Key Specifications

Benchmark Performance

Benchmark	Score	Unit	Evaluated At	Notes	Source
MMLU	83.9	%	2024-08-12	5-shot	view
HUMANEVAL	89	pass@1	2024-08-12	—	view
GSM8K	88.9	%	2024-08-12	0-shot CoT	view
MATH	65.7	%	2024-08-12	0-shot CoT	view
BBH	82.1	%	2024-08-12	3-shot CoT	view
GPQA	40.5	%	2024-08-12	0-shot	view
IFEVAL	78.2	%	2024-08-12	prompt_strict	view
ARC	94.5	%	2024-08-12	challenge	view
MUSR	57.7	%	2024-08-12	0-shot	view
WINOGRANDE	86.5	%	2024-08-12	0-shot	view

Name: Hermes 3 Llama 3.1 405B
Brand: other

NousResearch Hermes 3 Llama 3.1 405B 微调模型, 131K 上下文, 改进中立性与推理, 性能超越基座模型。

공급업체	버전	출시일	컨텍스트 창	입력 모달리티	출력 모달리티	라이선스
Other	hermes-3-llama-3-1-405b	2024-08-12	131K	text	text	Llama 3.1 Community License

벤치마크	점수	단위	비고
MMLU (Massive Multitask Language Understanding)	83.9	%	5-shot
HumanEval	89.0	pass@1	—
GSM8K (Grade School Math 8K)	88.9	%	0-shot CoT
MATH	65.7	%	0-shot CoT
BBH (BIG-Bench Hard)	82.1	%	3-shot CoT
GPQA	40.5	%	0-shot
IFEval	78.2	%	prompt_strict
ARC	94.5	%	challenge
MUSR	57.7	%	0-shot
WinoGrande	86.5	%	0-shot

입력	출력	캐시 읽기	캐시 쓰기
—	—	—	—

백만 토큰당