Instruction Following Leaderboard

Reset
Models IFLONG26
ACHAP-CollarF1 ACHAP-COMET ACHAP-GC-BERTScore ACHAP-TM-BERTScore ACHAP-TM-MATCHED ACHAP-WER ASR-WER QA-BERTScore QE-accuracy QE-format-accuracy SUM-BERTScore TRANS-COMET
Phi4-Multimodal 0.104 0.248 0.666 0.749 0.224 0.985 0.820 0.402 0.394 0.253 0.170 0.411
Qwen3-Omni 0.304 0.283 0.708 0.741 0.381 0.552 0.540 0.294 0.454 0.500 0.103 0.446
FBK_IWSLT26_IF_LONG_ CONSTRAINED_PRIMARY 0.000 0.523 0.604 0.801 0.130 0.090 0.049 0.376 0.290 0.351 0.201 0.514
FBK_IWSLT26_IF_LONG_ CONSTRAINED_CONTRASTIVE_HYB 0.000 0.516 0.611 0.799 0.132 0.050 0.032 0.368 0.290 0.351 0.202 0.526
KIT_IWSLT26_IF_LONG_ unconstrained_ contrastive1 0.421 0.532 0.693 0.743 0.467 0.023 0.016 0.344 0.361 0.498 0.268 0.632
FBK_IWSLT26_IF_LONG_ CONSTRAINED_CONTRASTIVE_VAD 0.000 0.513 0.625 0.805 0.135 0.050 0.044 0.369 0.290 0.351 0.202 0.531
KIT_IWSLT26_IF_ LONG_ unconstrained_ primary 0.474 0.566 0.737 0.772 0.525 0.078 0.067 0.427 0.250 0.001 0.275 0.564
Models IFSHORT26
ACHAP-CollarF1 ACHAP-COMET ACHAP-GC-BERTScore ACHAP-TM-BERTScore ACHAP-TM-MATCHED ACHAP-WER ASR-WER QA-BERTScore QE-accuracy QE-format-accuracy SUM-BERTScore TRANS-COMET
Phi4-Multimodal N/A N/A N/A N/A N/A N/A 0.767 0.421 0.394 0.253 0.000 0.601
Qwen3-Omni N/A N/A N/A N/A N/A N/A 0.549 0.325 0.454 0.500 0.000 0.635
KIT_IWSLT26_IF_SHORT_ unconstrained_ contrastive1 N/A N/A N/A N/A N/A N/A 0.042 0.448 0.361 0.498 N/A 0.626
FBK_IWSLT26_IF_SHORT_ CONSTRAINED_CONTRASTIVE N/A N/A N/A N/A N/A N/A 0.036 0.496 0.290 0.351 N/A 0.551
NLE_IWSLT26_IF_SHORT_CONSTRAINED_PRIMARY N/A N/A N/A N/A N/A N/A 0.034 0.486 0.420 0.499 N/A 0.580
BSC_IWSLT26_IF_SHORT_unconstrained_primary N/A N/A N/A N/A N/A N/A 0.034 0.440 0.429 0.476 N/A 0.591
NLE_IWSLT26_IF_SHORT_UNCONSTRAINED_CONSTRASTIVE N/A N/A N/A N/A N/A N/A 0.033 0.486 0.208 0.005 N/A 0.559
BSC_IWSLT26_IF_SHORT_unconstrained_contrastive3 N/A N/A N/A N/A N/A N/A 0.032 0.299 0.199 0.178 N/A 0.397
BSC_IWSLT26_IF_SHORT_unconstrained_contrastive N/A N/A N/A N/A N/A N/A 0.032 0.400 0.421 0.415 N/A 0.584
FBK_IWSLT26_IF_SHORT_ CONSTRAINED_PRIMARY N/A N/A N/A N/A N/A N/A 0.031 0.507 0.419 0.484 N/A 0.570
KIT_IWSLT26_IF_ SHORT_unconstrained_ primary N/A N/A N/A N/A N/A N/A 0.019 0.484 0.000 0.000 N/A 0.634