GPT-3.5 Turbo 16K: Complete Benchmark Performance Guide

openai 3.5-turbo-16k 2023-06-13 Proprietary

In-depth analysis of GPT-3.5 Turbo 16K performance on MMLU, HumanEval, GSM8K, MATH, BBH benchmarks, including pricing and use cases.

Key Specifications

Vendoropenai
Version3.5-turbo-16k
Release Date2023-06-13
Context Window16385 tokens
Input Modalitiestext
Output Modalitiestext
LicenseProprietary
Documentationhttps://platform.openai.com/docs/models

Benchmark Performance

BenchmarkScoreUnitEvaluated AtNotesSource
MMLU64.2%2023-06-135-shotview
HUMANEVAL36.8pass@12023-06-13—view
GSM8K52.8%2023-06-130-shot CoTview
MATH30.4%2023-06-130-shot CoTview
BBH50.2%2023-06-133-shot CoTview
GPQA21.6%2023-06-130-shotview
IFEVAL48.5%2023-06-13prompt_strictview
ARC83.1%2023-06-13challengeview
MUSR29.1%2023-06-130-shotview
WINOGRANDE74%2023-06-130-shotview

Pricing

TierPriceCurrency
Input$3 / MtokUSD
Output$4 / MtokUSD
Cache Read$0 / MtokUSD
Cache Write$0 / MtokUSD

Source: https://openai.com/api/pricing/ · as of 2023-06-13

Compliance

  • Data Residency: US
  • SOC2: ✓
  • HIPAA: ✓
  • GDPR: ✓
  • ISO 27001: ✓

GPT-3.5 Turbo 16K

모델 개요

OpenAI GPT-3.5 Turbo 16K 长上下文版本, 支持 16385 token, 适合中等长度文档处理。

핵심 사양

공급업체버전출시일컨텍스트 창입력 모달리티출력 모달리티라이선스
Openai3.5-turbo-16k2023-06-1316KtexttextProprietary

벤치마크 성능

벤치마크점수단위비고
MMLU (Massive Multitask Language Understanding)64.2%5-shot
HumanEval36.8pass@1—
GSM8K (Grade School Math 8K)52.8%0-shot CoT
MATH30.4%0-shot CoT
BBH (BIG-Bench Hard)50.2%3-shot CoT
GPQA21.6%0-shot
IFEval48.5%prompt_strict
ARC83.1%challenge
MUSR29.1%0-shot
WinoGrande74.0%0-shot

가격

입력출력캐시 읽기캐시 쓰기
————

백만 토큰당

강점

  • 上下文窗口 16K，支持长文本。

약점

  • HumanEval 36.8，代码能力较弱。
  • 闭源专有模型，不支持自托管。
  • 上下文窗口 16K 偏小。

사용 사례

  • 长文档摘要

참고문헌