Vocal Affect Bench
How accurately can models identify emotions expressed in speech? We measure accuracy as the % of scored clips where the expressed emotion is identified correctly, across seven emotion categories.
- Benchmark
- 280 recordings · private evaluation set
- Results published
Methodology
Private evaluation dataset: 280 recordings. Audio, reference labels and detailed predictions remain private.
Models receive audio and predict the expressed emotion. Scores report accuracy overall and by emotion.
Evaluation set and public release
Public test-only release: 280 human-recorded English WAV clips, 2.32 hours, 16 kHz mono, balanced across seven expressed emotions. Browse the public release ↗
Public release under a custom license: non-audio materials/software are MIT licensed; audio prohibits voice cloning and requires the copyright notice and license with copies. See the full license. Dataset license ↗
Price and speed
Price is a reference API estimate in USD per audio hour for model inference, excluding verifier charges, infrastructure, credits, discounts and tax. Unpriced or incomplete costs remain unreported. Speed is audio duration divided by total inference call time, including adapter processing, transport and queue waits; it is independent of evaluation concurrency. Streaming includes audio uploaded at 1× speed. Higher Accuracy and speed are better; lower price is better. Models where we cannot estimate price are omitted from the comparison.
Change logLatest recorded change
Model additions, withdrawals, and changes to scoring or reporting. Historical scores refer to the evaluation set used at the time.
- Updated
Added target highlights and Pareto frontiers to quality, price and speed comparisons.
- Updated
- Methodology
Replaced public-release standings with results on a separate private 280-recording evaluation set for nine models: Qwen3.5 Omni+, Gemini 3.6 Flash, Gemini 3.8 Flash, Hopper Audio Perception-1, Inworld Voice Profile, Voxtral Small, OpenAI Realtime, Oruk Resonance 2, and Thinking Machines Inkling.
- Retired
Removed Hume Prosody from current standings after its API retirement. Replaced the Gemini 3.5 Flash entry with a Gemini 3.6 Flash evaluation.
- Updated
Published refreshed results for Hopper, Oruk, Gemini 3.8 Flash, Inworld, OpenAI Realtime, Inkling, Qwen and Voxtral. Qwen and Voxtral now use direct provider evaluations, and Inkling uses Tinker.
- Added
Added inference price and speed measurements, with quality, price and speed comparison charts. Unavailable prices remain unreported.
- Methodology
- Published
Introduced a dedicated interactive leaderboard with model comparisons and emotion breakdowns.
- Published
- Added
Added Hopper Audio Perception-1 to the public-release benchmark.
- Added
- Added
Added Oruk Resonance 2 and Gemini 3.8 Flash to the public-release benchmark.
- Added
- Added
Added Thinking Machines Inkling to the public-release benchmark.
- Added
- Published
Added the first VocalAffectBench report and dashboard: 280 clips across seven emotions, with six baseline models: Gemini 3.5 Flash, Hume Prosody, Qwen3.5 Omni+, Voxtral Small, Inworld Voice, and OpenAI Realtime.
- Published