Hugging Face has launched an open-source text-to-speech evaluation leaderboard designed to address the fragmentation and inconsistency plaguing TTS model assessment. While arena-based leaderboards like TTS Arena v2 and Artificial Analysis have become community standards, they struggle to keep pace with the explosive growth in TTS model releases—there are now over 8,000 TTS models on the Hugging Face Hub as of September 2026. Additionally, open-source models remain underrepresented on existing platforms, with only 16 of 92 models on Artificial Analysis being open-weights, partly because commercial providers have greater incentive to seek placement than open-source researchers.
The new Open TTS Leaderboard takes a different approach by employing objective, scalable metrics rather than relying solely on human preference votes. Models are evaluated on intelligibility using word and character error rates (WER/CER) powered by Qwen3 ASR, inference speed through real-time factor (RTFx) and time-to-first-audio (TTFA) measurements on H200 GPUs, and voice preservation through speaker similarity scores computed from WavLM embeddings. This objective methodology reduces evaluation turnaround from weeks for voting-based systems to just hours, enabling the leaderboard to keep pace with rapid model releases.
The leaderboard emphasizes multilingual support and voice cloning capabilities, with toggles for evaluating performance across multiple languages and scenarios where reference audio is provided. A "Listen" tab lets users compare model outputs directly and vote on preferences, which Hugging Face plans to incorporate into future iterations. The team emphasized that objective metrics complement rather than replace human preference evaluation, and they are positioning the leaderboard as a community-shaped resource open to feedback.
Key Points
Hugging Face launched an open TTS leaderboard using objective metrics to evaluate models at scale, reducing evaluation time from weeks to hours
Open-source models have been underrepresented on existing arena-based leaderboards due to operational costs and incentive factors
The leaderboard evaluates models on intelligibility (WER/CER), speed (RTFx/TTFA), and speaker similarity while supporting multilingual evaluation and voice cloning
Community voting on model outputs via the "Listen" tab will inform future leaderboard rankings alongside objective metrics