NLE_IWSLT26_IF_SHORT_CONSTRAINED_PRIMARY

DATA
This model is trained on all available constrained data (EuroParl, LibriSQA, CoVoST2, GigaST) and on synthetic datasets. We generate synthetic translation and question-answering data for all datasets using SeamlessM4T, followed by COMET-based quality filtering.
Additionally, we created a synthetic dataset that we included in the last two stages of training, and that we name FakeACL.
Inspired by the MCIF validation dataset and  ACL60/60 from previous year, FakeACL consists of 21,400 sentence-level segments simulating NLP paper presentations scripted by Qwen3-4b-Instruct-2507. The audio is synthesized by SeamlessM4T.
From these segments, we also generate 38,968 question-answer pairs using the same LLM. The resulting dataset is then translated using the same SeamlessM4T + COMET filtering pipeline. This data is leveraged across multiple tasks, including speech translation (ST), spoken question answering (SQA), machine translation (MT), and text-based question answering (QA).

BACKBONES
SeamlessM4T v2 Large

Qwen3-4B-Instruct-2507

TRAINING
Our training procedure closely follows last year’s submission (see: https://aclanthology.org/2025.iwslt-1.17/)
1. Speech-to-LLM-Embedding projector: The projector corresponds to an updated version of speechmapper (https://arxiv.org/abs/2601.20417) trained to mimic the text embeddings of Qwen3-4b-Instruct-2507. The model receives as input the SeamlessM4T final encoder layer's representation (averaged every two frames). It is trained on ASR-only data. 
2. Text adaptation: Lora adapters are trained from text as last year, as we notice this year's LLM is much more sensitive to noise and difficult to control without lora adapters.
3. Alignment stage: A final quick 3k steps training step (~2.5h of training approximately) aligns speech-only projector and text-only lora adapters for the target tasks.