Mistral Large 2: Complete Benchmark Performance Guide

mistral large-2 2024-07-24 Mistral Research License

In-depth analysis of Mistral Large 2 performance on MMLU, HumanEval, GSM8K, MATH, BBH benchmarks, including pricing and use cases.

Key Specifications

Vendormistral
Versionlarge-2
Release Date2024-07-24
Context Window128000 tokens
Input Modalitiestext
Output Modalitiestext
LicenseMistral Research License
Documentationhttps://docs.mistral.ai/

Benchmark Performance

BenchmarkScoreUnitEvaluated AtNotesSource
MMLU84%2024-07-245-shotview
HUMANEVAL92pass@12024-07-24—view
GSM8K93%2024-07-240-shot CoTview
MATH71%2024-07-240-shot CoTview
BBH81%2024-07-243-shot CoTview

Pricing

TierPriceCurrency
Input$2 / MtokUSD
Output$6 / MtokUSD
Cache Read$0 / MtokUSD
Cache Write$0 / MtokUSD

Source: https://mistral.ai/technology/ · as of 2024-08-01

Compliance

  • Data Residency: EU
  • SOC2: ✓
  • HIPAA: ✗
  • GDPR: ✓
  • ISO 27001: ✓

Mistral Large 2

모델 개요

Mistral Large 2 是 Mistral AI 旗舰模型，123B 参数，128K 上下文窗口，在代码生成、数学推理与多语言任务上表现强劲，支持欧盟数据驻留与 GDPR/ISO 27001 合规。

핵심 사양

공급업체버전출시일컨텍스트 창입력 모달리티출력 모달리티라이선스
Mistrallarge-22024-07-24128KtexttextMistral Research License

벤치마크 성능

벤치마크점수단위비고
MMLU (Massive Multitask Language Understanding)84.0%5-shot
HumanEval92.0pass@1—
GSM8K (Grade School Math 8K)93.0%0-shot CoT
MATH71.0%0-shot CoT
BBH (BIG-Bench Hard)81.0%3-shot CoT

가격

입력출력캐시 읽기캐시 쓰기
————

백만 토큰당

강점

  • MMLU score 84.0, strong knowledge reasoning.
  • HumanEval 92.0, excellent code generation.
  • GSM8K 93.0, robust math reasoning.

약점

  • 闭源专有模型，不支持自托管。

사용 사례

  • 代码生成与调试
  • Agent 工作流与工具调用

참고문헌