Command R+: Complete Benchmark Performance Guide

cohere r-plus 2024-04-04 CC-BY-NC-4.0

In-depth analysis of Command R+ performance on MMLU, HumanEval, GSM8K, MATH, BBH benchmarks, including pricing and use cases.

Key Specifications

Vendorcohere
Versionr-plus
Release Date2024-04-04
Context Window128000 tokens
Input Modalitiestext
Output Modalitiestext
LicenseCC-BY-NC-4.0
Documentationhttps://docs.cohere.com/docs/command-r-plus

Benchmark Performance

BenchmarkScoreUnitEvaluated AtNotesSource
MMLU75%2024-04-045-shotview
HUMANEVAL70.7pass@12024-04-04—view
GSM8K68.4%2024-04-040-shot CoTview
MATH35.6%2024-04-040-shot CoTview
BBH66.1%2024-04-043-shot CoTview

Pricing

TierPriceCurrency
Input$2.5 / MtokUSD
Output$10 / MtokUSD
Cache Read$0 / MtokUSD
Cache Write$0 / MtokUSD

Source: https://cohere.com/pricing · as of 2024-08-01

Compliance

  • Data Residency: US
  • SOC2: ✓
  • HIPAA: ✗
  • GDPR: ✓
  • ISO 27001: ✓

Command R+

모델 개요

Cohere Command R+ 是面向企业的 104B 参数模型，128K 上下文窗口，针对检索增强生成（RAG）与工具调用优化，支持 SOC2/GDPR/ISO 27001 合规，适合企业级生产部署。

핵심 사양

공급업체버전출시일컨텍스트 창입력 모달리티출력 모달리티라이선스
Coherer-plus2024-04-04128KtexttextCC-BY-NC-4.0

벤치마크 성능

벤치마크점수단위비고
MMLU (Massive Multitask Language Understanding)75.0%5-shot
HumanEval70.7pass@1—
GSM8K (Grade School Math 8K)68.4%0-shot CoT
MATH35.6%0-shot CoT
BBH (BIG-Bench Hard)66.1%3-shot CoT

가격

입력출력캐시 읽기캐시 쓰기
————

백만 토큰당

강점

  • 企业级合规认证。

약점

  • 闭源专有模型，不支持自托管。

사용 사례

  • 代码生成与调试
  • 企业客户支持

참고문헌