Let's challenge speech models together!

Test
Compare
Improve speech models

SPEECHM is a platform for running and evaluating speech model challenges - using real-world data and open, trusted metrics.

Browse our challenges

Challenges:

Explore the core functions that enhance communication through language understanding, translation, and speech recognition.

  • Meetween

    The Meetween Challenge is a benchmarking initiative focused on evaluating progress in training SpeechLMM model versions, with the goal of advancing multimodal speech-language understanding and generation. It provides a shared task setting where Meetween project members can test, compare, and improve their models on curated datasets and evaluation metrics.

  • IWSLT 2025

    The International Conference on Spoken Language Translation (IWSLT) is an annual scientific conference, associated with shared tasks on spoken language translation, where both scientific papers and system descriptions are presented.

    The 22nd edition of IWSLT will be run as an ACL and ELRA sponsored event, co-located with ACL 2025 in Vienna, Austria on 31 July-1 August 2025. It will be run as a hybrid event.
See the whole list

Join a challenge

Ready to benchmark your models?

Join a challenge

Why SPEECHM?

Empowering open, fair, and practical evaluation of speech technolgies.

SPEECHM is a set of metrics and evaluation tools developed under the Meetween project (Horizon Europe).

The platform supports evaluation across tasks like ASR, ST, SLU, Summarization and more - with a focus on open sciece, transparency, and multilingual fairness.

  • No-code challenge creation
  • Trusted evaluation with open metrics
  • Compare models across organizations
  • Leaderboards & performance tracking

Powered by

  • Open Science
  • Github
  • Hugging Face

Partners