Yi 6B: Complete Benchmark Performance Guide

other yi-6b 2023-11-05 Apache 2.0

In-depth analysis of Yi 6B performance on MMLU, HumanEval, GSM8K, MATH, BBH benchmarks, including pricing and use cases.

Key Specifications

Vendorother
Versionyi-6b
Release Date2023-11-05
Context Window4096 tokens
Input Modalitiestext
Output Modalitiestext
LicenseApache 2.0
Documentationhttps://huggingface.co/models

Benchmark Performance

BenchmarkScoreUnitEvaluated AtNotesSource
MMLU68%2023-11-055-shotview
HUMANEVAL42.3pass@12023-11-05—view
GSM8K37%2023-11-050-shot CoTview
MATH29.9%2023-11-050-shot CoTview
BBH55.3%2023-11-053-shot CoTview
GPQA26.3%2023-11-050-shotview
IFEVAL54.9%2023-11-05prompt_strictview
ARC88.4%2023-11-05challengeview
MUSR29.5%2023-11-050-shotview
WINOGRANDE72.2%2023-11-050-shotview

Pricing

TierPriceCurrency
Input$0.18 / MtokUSD
Output$0.18 / MtokUSD
Cache Read$0 / MtokUSD
Cache Write$0 / MtokUSD

Source: https://huggingface.co/models · as of 2023-11-05

Compliance

  • Data Residency: self-host
  • SOC2: ✗
  • HIPAA: ✗
  • GDPR: ✗
  • ISO 27001: ✗

Yi 6B

모델 개요

01.AI Yi 6B 经济型首代模型, 4K 上下文, 6B 参数, 适合本地部署与研究用途。

핵심 사양

공급업체버전출시일컨텍스트 창입력 모달리티출력 모달리티라이선스
Otheryi-6b2023-11-054KtexttextApache 2.0

벤치마크 성능

벤치마크점수단위비고
MMLU (Massive Multitask Language Understanding)68.0%5-shot
HumanEval42.3pass@1—
GSM8K (Grade School Math 8K)37.0%0-shot CoT
MATH29.9%0-shot CoT
BBH (BIG-Bench Hard)55.3%3-shot CoT
GPQA26.3%0-shot
IFEval54.9%prompt_strict
ARC88.4%challenge
MUSR29.5%0-shot
WinoGrande72.2%0-shot

가격

입력출력캐시 읽기캐시 쓰기
————

백만 토큰당

강점

  • 可靠的通用模型。

약점

  • HumanEval 42.3，代码能力较弱。
  • 闭源专有模型，不支持自托管。
  • 上下文窗口 4K 偏小。

사용 사례

  • 通用对话与问答

참고문헌