Instruction Following Leaderboard
| Models | IFLONG26 | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ACHAP-CollarF1 | ACHAP-COMET | ACHAP-GC-BERTScore | ACHAP-TM-BERTScore | ACHAP-TM-MATCHED | ACHAP-WER | ASR-WER | QA-BERTScore | QE-accuracy | QE-format-accuracy | SUM-BERTScore | TRANS-COMET | |
| KIT_IWSLT26_IF_ LONG_ unconstrained_ primary 🏆 | 0.478 🏆 | 0.569 🏆 | 0.737 🏆 | 0.772 | 0.531 🏆 | 0.828 | 0.817 | 0.427 🏆 | 0.250 | 0.001 | 0.275 🏆 | 0.568 |
| KIT_IWSLT26_IF_LONG_ unconstrained_ contrastive1 | 0.430 | 0.536 | 0.701 | 0.743 | 0.476 | 0.773 🏆 | 0.766 🏆 | 0.344 | 0.361 | 0.498 | 0.268 | 0.632 🏆 |
| Qwen3-Omni | 0.308 | 0.317 | 0.724 | 0.745 | 0.387 | 0.802 | 0.790 | 0.294 | 0.454 🏆 | 0.500 🏆 | 0.103 | 0.478 |
| FBK_IWSLT26_IF_LONG_ CONSTRAINED_CONTRASTIVE_VAD | 0.000 | 0.526 | 0.625 | 0.805 🏆 | 0.135 | 0.800 | 0.794 | 0.369 | 0.290 | 0.351 | 0.202 | 0.539 |
| Phi4-Multimodal | 0.101 | 0.298 | 0.667 | 0.746 | 0.229 | 0.985 | 0.820 | 0.402 | 0.394 | 0.253 | 0.170 | 0.438 |
| FBK_IWSLT26_IF_LONG_ CONSTRAINED_PRIMARY | 0.000 | 0.531 | 0.604 | 0.801 | 0.130 | 0.840 | 0.799 | 0.376 | 0.290 | 0.351 | 0.201 | 0.524 |
| FBK_IWSLT26_IF_LONG_ CONSTRAINED_CONTRASTIVE_HYB | 0.000 | 0.528 | 0.611 | 0.799 | 0.132 | 0.800 | 0.782 | 0.368 | 0.290 | 0.351 | 0.202 | 0.534 |
| Models | IFSHORT26 | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ACHAP-CollarF1 | ACHAP-COMET | ACHAP-GC-BERTScore | ACHAP-TM-BERTScore | ACHAP-TM-MATCHED | ACHAP-WER | ASR-WER | QA-BERTScore | QE-accuracy | QE-format-accuracy | SUM-BERTScore | TRANS-COMET | |
| Qwen3-Omni 🏆 | N/A | N/A | N/A | N/A | N/A | N/A | 0.799 | 0.325 | 0.454 🏆 | 0.500 🏆 | N/A | 0.635 🏆 |
| FBK_IWSLT26_IF_SHORT_ CONSTRAINED_PRIMARY | N/A | N/A | N/A | N/A | N/A | N/A | 0.781 | 0.507 🏆 | 0.419 | 0.484 | N/A | 0.574 |
| BSC_IWSLT26_IF_SHORT_unconstrained_contrastive3 | N/A | N/A | N/A | N/A | N/A | N/A | 0.532 🏆 | 0.299 | 0.199 | 0.178 | N/A | 0.397 |
| BSC_IWSLT26_IF_SHORT_unconstrained_contrastive2 | N/A | N/A | N/A | N/A | N/A | N/A | 0.782 | 0.380 | 0.250 | 0.004 | N/A | 0.576 |
| NLE_IWSLT26_IF_SHORT_CONSTRAINED_PRIMARY | N/A | N/A | N/A | N/A | N/A | N/A | 0.784 | 0.486 | 0.420 | 0.499 | N/A | 0.585 |
| BSC_IWSLT26_IF_SHORT_unconstrained_primary | N/A | N/A | N/A | N/A | N/A | N/A | 0.784 | 0.440 | 0.429 | 0.476 | N/A | 0.600 |
| BSC_IWSLT26_IF_SHORT_unconstrained_contrastive | N/A | N/A | N/A | N/A | N/A | N/A | 0.782 | 0.400 | 0.421 | 0.415 | N/A | 0.586 |
| Phi4-Multimodal | N/A | N/A | N/A | N/A | N/A | N/A | 0.767 | 0.421 | 0.394 | 0.253 | N/A | 0.602 |
| FBK_IWSLT26_IF_SHORT_ CONSTRAINED_CONTRASTIVE | N/A | N/A | N/A | N/A | N/A | N/A | 0.786 | 0.496 | 0.290 | 0.351 | N/A | 0.559 |
| KIT_IWSLT26_IF_ SHORT_unconstrained_ primary | N/A | N/A | N/A | N/A | N/A | N/A | 0.769 | 0.484 | 0.000 | 0.000 | N/A | 0.634 |
| KIT_IWSLT26_IF_SHORT_ unconstrained_ contrastive1 | N/A | N/A | N/A | N/A | N/A | N/A | 0.792 | 0.448 | 0.361 | 0.498 | N/A | 0.626 |
| NLE_IWSLT26_IF_SHORT_UNCONSTRAINED_CONSTRASTIVE | N/A | N/A | N/A | N/A | N/A | N/A | 0.533 | 0.371 | 0.125 | 0.004 | N/A | 0.377 |