Instruction Following Leaderboard

Models IFLONG26
ACHAP-CollarF1 ACHAP-COMET ACHAP-GC-BERTScore ACHAP-TM-BERTScore ACHAP-TM-MATCHED ACHAP-WER ASR-WER QA-BERTScore QE-accuracy QE-format-accuracy SUM-BERTScore TRANS-COMET
KIT_IWSLT26_IF_ LONG_ unconstrained_ primary 🏆 0.478 🏆 0.569 🏆 0.737 🏆 0.772 0.531 🏆 0.828 0.817 0.427 🏆 0.250 0.001 0.275 🏆 0.568
KIT_IWSLT26_IF_LONG_ unconstrained_ contrastive1 0.430 0.536 0.701 0.743 0.476 0.773 🏆 0.766 🏆 0.344 0.361 0.498 0.268 0.632 🏆
Qwen3-Omni 0.308 0.317 0.724 0.745 0.387 0.802 0.790 0.294 0.454 🏆 0.500 🏆 0.103 0.478
FBK_IWSLT26_IF_LONG_ CONSTRAINED_CONTRASTIVE_VAD 0.000 0.526 0.625 0.805 🏆 0.135 0.800 0.794 0.369 0.290 0.351 0.202 0.539
Phi4-Multimodal 0.101 0.298 0.667 0.746 0.229 0.985 0.820 0.402 0.394 0.253 0.170 0.438
FBK_IWSLT26_IF_LONG_ CONSTRAINED_PRIMARY 0.000 0.531 0.604 0.801 0.130 0.840 0.799 0.376 0.290 0.351 0.201 0.524
FBK_IWSLT26_IF_LONG_ CONSTRAINED_CONTRASTIVE_HYB 0.000 0.528 0.611 0.799 0.132 0.800 0.782 0.368 0.290 0.351 0.202 0.534
Models IFSHORT26
ACHAP-CollarF1 ACHAP-COMET ACHAP-GC-BERTScore ACHAP-TM-BERTScore ACHAP-TM-MATCHED ACHAP-WER ASR-WER QA-BERTScore QE-accuracy QE-format-accuracy SUM-BERTScore TRANS-COMET
Qwen3-Omni 🏆 N/A N/A N/A N/A N/A N/A 0.799 0.325 0.454 🏆 0.500 🏆 N/A 0.635 🏆
FBK_IWSLT26_IF_SHORT_ CONSTRAINED_PRIMARY N/A N/A N/A N/A N/A N/A 0.781 0.507 🏆 0.419 0.484 N/A 0.574
BSC_IWSLT26_IF_SHORT_unconstrained_contrastive3 N/A N/A N/A N/A N/A N/A 0.532 🏆 0.299 0.199 0.178 N/A 0.397
BSC_IWSLT26_IF_SHORT_unconstrained_contrastive2 N/A N/A N/A N/A N/A N/A 0.782 0.380 0.250 0.004 N/A 0.576
NLE_IWSLT26_IF_SHORT_CONSTRAINED_PRIMARY N/A N/A N/A N/A N/A N/A 0.784 0.486 0.420 0.499 N/A 0.585
BSC_IWSLT26_IF_SHORT_unconstrained_primary N/A N/A N/A N/A N/A N/A 0.784 0.440 0.429 0.476 N/A 0.600
BSC_IWSLT26_IF_SHORT_unconstrained_contrastive N/A N/A N/A N/A N/A N/A 0.782 0.400 0.421 0.415 N/A 0.586
Phi4-Multimodal N/A N/A N/A N/A N/A N/A 0.767 0.421 0.394 0.253 N/A 0.602
FBK_IWSLT26_IF_SHORT_ CONSTRAINED_CONTRASTIVE N/A N/A N/A N/A N/A N/A 0.786 0.496 0.290 0.351 N/A 0.559
KIT_IWSLT26_IF_ SHORT_unconstrained_ primary N/A N/A N/A N/A N/A N/A 0.769 0.484 0.000 0.000 N/A 0.634
KIT_IWSLT26_IF_SHORT_ unconstrained_ contrastive1 N/A N/A N/A N/A N/A N/A 0.792 0.448 0.361 0.498 N/A 0.626
NLE_IWSLT26_IF_SHORT_UNCONSTRAINED_CONSTRASTIVE N/A N/A N/A N/A N/A N/A 0.533 0.371 0.125 0.004 N/A 0.377