SLT Results for the Spanish to English direction
(in red the best primary system for each metric)
FTE Task
| SYSTEM - Scoring |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_system1 |
10.4904 |
0.4816 |
10.5898 |
0.4915 |
0.4816 |
41.6838 |
41.0468 |
30.1754 |
29.2495 |
0.4919 |
0.4955 |
| IBM_PRIMARY_system2 |
10.4658 |
0.4803 |
10.5659 |
0.4903 |
0.4803 |
41.8422 |
41.2017 |
30.2572 |
29.3348 |
0.4935 |
0.4958 |
| IBM_PRIMARY_system3 |
10.4587 |
0.4799 |
10.5570 |
0.4897 |
0.4799 |
41.8439 |
41.2121 |
30.3199 |
29.3992 |
0.4920 |
0.4948 |
| IBM_PRIMARY_system4 |
10.4764 |
0.4809 |
10.5749 |
0.4907 |
0.4809 |
41.7934 |
41.1616 |
30.2555 |
29.3348 |
0.4928 |
0.4956 |
| IRST-PRIMARY |
10.2231 |
0.4619 |
10.4436 |
0.4786 |
0.4611 |
43.3564 |
42.3400 |
31.3433 |
29.9039 |
0.4903 |
0.4974 |
| IRST-SECONDARY-1 |
10.1443 |
0.4558 |
10.3656 |
0.4726 |
0.4539 |
43.6610 |
42.6219 |
31.6619 |
30.1998 |
0.4871 |
0.4950 |
| IRST-SECONDARY-2 |
10.1318 |
0.4555 |
10.3532 |
0.4722 |
0.4551 |
43.7393 |
42.7142 |
31.6236 |
30.1650 |
0.4877 |
0.4941 |
| ROVER |
10.5049 |
0.4807 |
10.7760 |
0.5006 |
0.4807 |
41.6420 |
40.3993 |
30.0258 |
28.3827 |
0.4969 |
0.5019 |
| RWTH-PRIMARY_1 |
10.3599 |
0.4711 |
10.4457 |
0.4799 |
0.4712 |
42.8900 |
42.3347 |
30.9308 |
30.1319 |
0.5010 |
0.5009 |
| RWTH-PRIMARY_2 |
10.2153 |
0.4605 |
10.3122 |
0.4701 |
0.4606 |
43.2938 |
42.7089 |
31.3520 |
30.4870 |
0.4965 |
0.4966 |
| UED-PRIMARY |
10.1137 |
0.4559 |
10.3341 |
0.4724 |
0.4560 |
43.7428 |
42.7211 |
31.6688 |
30.1911 |
0.4926 |
0.4936 |
| UKA-PRIMARY_1 |
9.6290 |
0.4123 |
9.9160 |
0.4309 |
0.4098 |
47.1682 |
45.7218 |
33.6373 |
31.7123 |
0.4710 |
0.4778 |
| UPC-PRIMARY-1 |
10.2955 |
0.4645 |
10.4497 |
0.4772 |
0.4646 |
42.5471 |
41.7273 |
30.9656 |
29.8343 |
0.4897 |
0.4945 |
| UPC-PRIMARY-2 |
10.2455 |
0.4575 |
10.3797 |
0.4693 |
0.4529 |
42.7455 |
42.0423 |
31.1484 |
30.1511 |
0.4779 |
0.4874 |
| USaar-Primary-Ptolemaios |
9.0557 |
0.3724 |
9.4984 |
0.3997 |
0.3724 |
63.1549 |
61.7137 |
34.9462 |
32.3145 |
0.4648 |
0.4692 |
| UW-PRIMARY-system1 |
10.0941 |
0.4563 |
10.3227 |
0.4723 |
0.4563 |
44.0631 |
42.9944 |
31.7419 |
30.2294 |
0.4895 |
0.4906 |
| UW-PRIMARY-system2 |
10.1186 |
0.4599 |
10.3479 |
0.4764 |
0.4599 |
43.8995 |
42.8186 |
31.8168 |
30.2886 |
0.4948 |
0.4955 |
| SYSTRAN** |
9.4487 |
0.4057 |
9.4594 |
0.4094 |
0.4057 |
47.2743 |
46.9593 |
34.5755 |
34.0237 |
0.4326 |
0.4268 |
*cs: case-sensitive
**translation with a commercial cd
| SYSTEM - Ranking |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_system1 |
2 |
1 |
2 |
2 |
1 |
2 |
2 |
2 |
2 |
9 |
7 |
| IBM_PRIMARY_system2 |
4 |
4 |
4 |
4 |
4 |
4 |
4 |
4 |
3 |
5 |
5 |
| IBM_PRIMARY_system3 |
5 |
5 |
5 |
5 |
5 |
5 |
5 |
5 |
5 |
8 |
10 |
| IBM_PRIMARY_system4 |
3 |
2 |
3 |
3 |
2 |
3 |
3 |
3 |
3 |
6 |
6 |
| IRST-PRIMARY |
9 |
8 |
8 |
7 |
8 |
10 |
9 |
9 |
7 |
10 |
3 |
| IRST-SECONDARY-1 |
11 |
14 |
10 |
10 |
14 |
11 |
10 |
12 |
12 |
14 |
9 |
| IRST-SECONDARY-2 |
12 |
15 |
11 |
13 |
13 |
12 |
12 |
11 |
10 |
13 |
12 |
| ROVER |
1 |
3 |
1 |
1 |
3 |
1 |
1 |
1 |
1 |
2 |
1 |
| RWTH-PRIMARY_1 |
6 |
6 |
7 |
6 |
6 |
8 |
8 |
6 |
8 |
1 |
2 |
| RWTH-PRIMARY_2 |
10 |
9 |
15 |
14 |
9 |
9 |
11 |
10 |
15 |
3 |
4 |
| UED-PRIMARY |
14 |
13 |
13 |
11 |
12 |
13 |
13 |
13 |
11 |
7 |
13 |
| UKA-PRIMARY_1 |
16 |
16 |
16 |
16 |
16 |
16 |
16 |
16 |
16 |
16 |
16 |
| UPC-PRIMARY-1 |
7 |
7 |
6 |
8 |
7 |
6 |
6 |
7 |
6 |
11 |
11 |
| UPC-PRIMARY-2 |
8 |
11 |
9 |
15 |
15 |
7 |
7 |
8 |
9 |
15 |
15 |
| USaar-Primary-Ptolemaios |
18 |
18 |
17 |
18 |
18 |
18 |
18 |
18 |
17 |
17 |
17 |
| UW-PRIMARY-system1 |
15 |
12 |
14 |
12 |
11 |
15 |
15 |
14 |
13 |
12 |
14 |
| UW-PRIMARY-system2 |
13 |
10 |
12 |
9 |
10 |
14 |
14 |
15 |
14 |
4 |
7 |
| SYSTRAN** |
17 |
17 |
18 |
17 |
17 |
17 |
17 |
17 |
18 |
18 |
18 |
*cs: case-sensitive
**translation with a commercial cd
VERBATIM Task
| SYSTEM - Scoring |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_system1 |
11.039 |
0.5254 |
11.1259 |
0.5338 |
0.5241 |
37.4561 |
36.7391 |
26.9847 |
26.2549 |
0.5278 |
0.5388 |
| IBM_PRIMARY_system2 |
11.0487 |
0.5262 |
11.1347 |
0.5344 |
0.5252 |
37.4338 |
36.7232 |
26.9385 |
26.2103 |
0.5282 |
0.5391 |
| IBM_PRIMARY_system3 |
11.0328 |
0.5250 |
11.1184 |
0.5332 |
0.5241 |
37.4943 |
36.7853 |
26.9974 |
26.2709 |
0.5273 |
0.5382 |
| IBM_PRIMARY_system4 |
11.0536 |
0.5264 |
11.1399 |
0.5346 |
0.5252 |
37.4083 |
36.6961 |
26.9082 |
26.1785 |
0.5280 |
0.5391 |
| IRST-PRIMARY ** |
10.5715 |
0.4885 |
10.8500 |
0.5072 |
0.4849 |
39.9353 |
38.7786 |
28.6609 |
27.3384 |
0.5096 |
0.5242 |
| IRST-SECONDARY-1 |
10.2742 |
0.4716 |
10.7762 |
0.5019 |
0.4663 |
41.8856 |
39.1785 |
30.7067 |
27.8610 |
0.5050 |
0.5213 |
| IRST-SECONDARY-2 |
10.2734 |
0.4717 |
10.7771 |
0.5024 |
0.4680 |
41.8107 |
39.0861 |
30.5442 |
27.6969 |
0.5075 |
0.5225 |
| LIMSI-PRIMARY |
9.7154 |
0.4259 |
9.8347 |
0.4361 |
0.4208 |
44.5002 |
43.7274 |
31.7631 |
30.9569 |
0.4610 |
0.4774 |
| ROVER |
11.0913 |
0.5255 |
11.4285 |
0.5486 |
0.5208 |
36.8203 |
35.2796 |
26.7823 |
25.1237 |
0.5273 |
0.5436 |
| RWTH-PRIMARY_1 |
11.0958 |
0.5245 |
11.2173 |
0.5353 |
0.5191 |
37.7285 |
36.9701 |
27.4117 |
26.6134 |
0.5188 |
0.5365 |
| RWTH-PRIMARY_2 |
10.9791 |
0.5157 |
11.1134 |
0.5274 |
0.5120 |
37.8863 |
37.0848 |
27.5232 |
26.6724 |
0.5185 |
0.5330 |
| RWTH-PRIMARY_3 |
10.8673 |
0.5044 |
11.0029 |
0.5164 |
0.5034 |
38.7786 |
37.9660 |
27.7128 |
26.8365 |
0.5174 |
0.5285 |
| UKA-PRIMARY_1 |
9.8926 |
0.4318 |
10.3708 |
0.4609 |
0.4284 |
44.8492 |
42.3715 |
31.5416 |
28.8154 |
0.4828 |
0.4968 |
| UPC-PRIMARY |
10.6546 |
0.4963 |
10.8183 |
0.5080 |
0.4957 |
39.6007 |
38.7801 |
28.8473 |
27.9726 |
0.5070 |
0.5195 |
| USaar-Primary-Ptolemaios |
9.0939 |
0.3750 |
9.6914 |
0.4063 |
0.3742 |
68.6073 |
66.2954 |
33.5301 |
30.0869 |
0.4661 |
0.4786 |
| UW-PRIMARY-system1 |
9.8971 |
0.4497 |
10.3563 |
0.4770 |
0.4497 |
44.3696 |
41.8393 |
32.2809 |
29.5946 |
0.5053 |
0.5132 |
| UW-PRIMARY-system2 |
9.9018 |
0.4507 |
10.3626 |
0.4782 |
0.4507 |
44.2995 |
41.7517 |
32.3080 |
29.6185 |
0.5061 |
0.5140 |
| SYSTRAN*** |
9.8901 |
0.4373 |
9.8988 |
0.4400 |
0.4374 |
43.5394 |
43.2590 |
31.1895 |
30.8549 |
0.4576 |
0.4591 |
*cs: case-sensitive
**submitted the 2006/03/29
***translation with a commercial cd
| SYSTEM - Ranking |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_system1 |
5 |
4 |
5 |
5 |
3 |
4 |
4 |
4 |
4 |
3 |
4 |
| IBM_PRIMARY_system2 |
4 |
2 |
4 |
4 |
1 |
3 |
3 |
3 |
3 |
1 |
2 |
| IBM_PRIMARY_system3 |
6 |
5 |
6 |
6 |
3 |
5 |
5 |
5 |
5 |
4 |
5 |
| IBM_PRIMARY_system4 |
3 |
1 |
3 |
3 |
1 |
2 |
2 |
2 |
2 |
2 |
2 |
| IRST-PRIMARY ** |
10 |
10 |
9 |
10 |
10 |
10 |
9 |
9 |
9 |
9 |
9 |
| IRST-SECONDARY-1 |
11 |
12 |
12 |
12 |
12 |
12 |
12 |
12 |
11 |
14 |
11 |
| IRST-SECONDARY-2 |
12 |
11 |
11 |
11 |
11 |
11 |
11 |
11 |
10 |
10 |
10 |
| LIMSI-PRIMARY |
17 |
17 |
17 |
17 |
17 |
16 |
17 |
15 |
18 |
17 |
17 |
| ROVER |
2 |
3 |
1 |
1 |
5 |
1 |
1 |
1 |
1 |
4 |
1 |
| RWTH-PRIMARY_1 |
1 |
6 |
2 |
2 |
6 |
6 |
6 |
6 |
6 |
6 |
6 |
| RWTH-PRIMARY_2 |
7 |
7 |
7 |
7 |
7 |
7 |
7 |
7 |
7 |
7 |
7 |
| RWTH-PRIMARY_3 |
8 |
8 |
8 |
8 |
8 |
8 |
8 |
8 |
8 |
8 |
8 |
| UKA-PRIMARY_1 |
15 |
16 |
13 |
15 |
16 |
17 |
15 |
14 |
13 |
15 |
15 |
| UPC-PRIMARY |
9 |
9 |
10 |
9 |
9 |
9 |
10 |
10 |
12 |
11 |
12 |
| USaar-Primary-Ptolemaios |
18 |
18 |
18 |
18 |
18 |
18 |
18 |
18 |
16 |
16 |
16 |
| UW-PRIMARY-system1 |
14 |
14 |
15 |
14 |
14 |
15 |
14 |
16 |
14 |
13 |
14 |
| UW-PRIMARY-system2 |
13 |
13 |
14 |
13 |
13 |
14 |
13 |
17 |
15 |
12 |
13 |
| SYSTRAN*** |
16 |
15 |
16 |
16 |
15 |
13 |
16 |
13 |
17 |
18 |
18 |
*cs: case-sensitive
**submitted the 2006/03/29
***translation with a commercial cd
ASR Task
| SYSTEM - Scoring |
AS-WER |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_1 |
47.4274 |
9.5704 |
0.3941 |
9.7589 |
0.4058 |
0.3837 |
48.7299 |
47.6461 |
34.7222 |
33.5529 |
0.4976 |
0.5118 |
| IBM_PRIMARY_2 |
48.0344 |
9.5114 |
0.3906 |
9.7004 |
0.4026 |
0.3815 |
49.4064 |
48.3241 |
34.7658 |
33.6088 |
0.4993 |
0.5115 |
| IBM_PRIMARY_3 |
48.1232 |
9.4991 |
0.3901 |
9.6863 |
0.402 |
0.3808 |
49.4795 |
48.4096 |
34.8544 |
33.7068 |
0.4982 |
0.5105 |
| IBM_PRIMARY_4 |
47.4463 |
9.5766 |
0.3961 |
9.7663 |
0.4078 |
0.3851 |
48.7828 |
47.6989 |
34.6802 |
33.4907 |
0.4997 |
0.5128 |
| IRST-PRIMARY** |
49.0393 |
9.0311 |
0.343 |
9.2759 |
0.3569 |
0.3392 |
50.498 |
49.3411 |
38.5756 |
37.2927 |
0.4825 |
0.5015 |
| IRST-SECONDARY-1 |
49.4264 |
8.6367 |
0.3252 |
9.1284 |
0.3507 |
0.3224 |
52.5211 |
49.6956 |
40.8211 |
37.8494 |
0.476 |
0.4978 |
| IRST-SECONDARY-2 |
49.2655 |
8.6973 |
0.3282 |
9.1854 |
0.3537 |
0.3246 |
52.3936 |
49.5805 |
40.5723 |
37.6364 |
0.4799 |
0.4999 |
| LIMSI-PRIMARY |
51.7206 |
8.4817 |
0.326 |
8.6489 |
0.3367 |
0.3213 |
52.5771 |
51.6347 |
38.1791 |
37.1636 |
0.4246 |
0.4563 |
| RWTH-PRIMARY_1 |
47.6568 |
9.2634 |
0.3613 |
9.4668 |
0.3731 |
0.3579 |
49.0021 |
47.8778 |
38.0204 |
36.8169 |
0.4832 |
0.5062 |
| RWTH-PRIMARY_2 |
48.2718 |
9.1748 |
0.3532 |
9.3775 |
0.3651 |
0.3487 |
49.6023 |
48.4796 |
38.4341 |
37.246 |
0.4837 |
0.5044 |
| UKA-PRIMARY_1 |
54.6320 |
8.4224 |
0.3013 |
8.8167 |
0.3208 |
0.2979 |
54.8225 |
52.6144 |
41.4229 |
39.0141 |
0.4527 |
0.4723 |
| UPC-PRIMARY |
49.5680 |
9.0411 |
0.3483 |
9.2824 |
0.3611 |
0.343 |
51.0112 |
49.8262 |
39.011 |
37.7063 |
0.4765 |
0.4931 |
| SYSTRAN*** |
62.1435 |
8.0623 |
0.2922 |
8.2167 |
0.3011 |
0.2922 |
62.268 |
61.3972 |
47.7705 |
46.7753 |
0.4481 |
0.4354 |
*cs: case-sensitive
**submitted the 2006/03/29
***translation with a commercial cd
| SYSTEM - Ranking |
AS-WER |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_1 |
1 |
2 |
2 |
2 |
2 |
2 |
1 |
1 |
2 |
2 |
4 |
2 |
| IBM_PRIMARY_2 |
4 |
3 |
3 |
3 |
3 |
3 |
4 |
4 |
3 |
3 |
2 |
3 |
| IBM_PRIMARY_3 |
5 |
4 |
4 |
4 |
4 |
4 |
5 |
5 |
4 |
4 |
3 |
4 |
| IBM_PRIMARY_4 |
2 |
1 |
1 |
1 |
1 |
1 |
2 |
2 |
1 |
1 |
1 |
1 |
| IRST-PRIMARY** |
7 |
8 |
8 |
8 |
8 |
8 |
7 |
7 |
8 |
8 |
7 |
7 |
| IRST-SECONDARY-1 |
9 |
10 |
11 |
10 |
10 |
10 |
10 |
9 |
11 |
11 |
10 |
9 |
| IRST-SECONDARY-2 |
8 |
9 |
9 |
9 |
9 |
9 |
9 |
8 |
10 |
9 |
8 |
8 |
| LIMSI-PRIMARY |
11 |
11 |
10 |
12 |
11 |
11 |
11 |
11 |
6 |
6 |
13 |
12 |
| RWTH-PRIMARY_1 |
3 |
5 |
5 |
5 |
5 |
5 |
3 |
3 |
5 |
5 |
6 |
5 |
| RWTH-PRIMARY_2 |
6 |
6 |
6 |
6 |
6 |
6 |
6 |
6 |
7 |
7 |
5 |
6 |
| UKA-PRIMARY_1 |
12 |
12 |
12 |
11 |
12 |
12 |
12 |
12 |
12 |
12 |
11 |
11 |
| UPC-PRIMARY |
10 |
7 |
7 |
7 |
7 |
7 |
8 |
10 |
9 |
10 |
9 |
10 |
| SYSTRAN*** |
13 |
13 |
13 |
13 |
13 |
13 |
13 |
13 |
13 |
13 |
12 |
13 |
*cs: case-sensitive
**submitted the 2006/03/29
***translation with a commercial cd
EPPS / FTE Task
| SYSTEM - Scoring |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_system1 |
10.7684 |
0.5406 |
10.8663 |
0.5504 |
0.5366 |
36.2092 |
35.6865 |
26.3712 |
25.6456 |
0.5159 |
0.5244 |
| IBM_PRIMARY_system2 |
10.7494 |
0.5392 |
10.8502 |
0.5493 |
0.5367 |
36.3055 |
35.7656 |
26.3746 |
25.6456 |
0.5171 |
0.5243 |
| IBM_PRIMARY_system3 |
10.7563 |
0.5399 |
10.8541 |
0.5496 |
0.5359 |
36.2470 |
35.7244 |
26.4159 |
25.6903 |
0.5154 |
0.5239 |
| IBM_PRIMARY_system4 |
10.7684 |
0.5406 |
10.8663 |
0.5504 |
0.5366 |
36.2092 |
35.6865 |
26.3712 |
25.6456 |
0.5159 |
0.5244 |
| IRST-PRIMARY |
10.5616 |
0.5240 |
10.7417 |
0.5389 |
0.5213 |
37.4299 |
36.6218 |
27.2068 |
26.1133 |
0.5187 |
0.5284 |
| IRST-SECONDARY-1 |
10.4772 |
0.5181 |
10.6579 |
0.5330 |
0.5144 |
37.7291 |
36.9004 |
27.6194 |
26.4950 |
0.5147 |
0.5252 |
| IRST-SECONDARY-2 |
10.4718 |
0.5175 |
10.6516 |
0.5323 |
0.5157 |
37.8254 |
37.0035 |
27.5059 |
26.3918 |
0.5159 |
0.5246 |
| ROVER |
10.7870 |
0.5399 |
11.0082 |
0.5582 |
0.5367 |
35.9891 |
35.0160 |
26.1270 |
24.8925 |
0.5213 |
0.5303 |
| RWTH-PRIMARY_1 |
10.6528 |
0.5310 |
10.7323 |
0.5396 |
0.5312 |
37.0551 |
36.5771 |
26.9110 |
26.2680 |
0.5290 |
0.5332 |
| RWTH-PRIMARY_2 |
10.5111 |
0.5201 |
10.6015 |
0.5295 |
0.5202 |
37.3302 |
36.8282 |
27.1758 |
26.4675 |
0.5254 |
0.5293 |
| UED-PRIMARY |
10.4792 |
0.5187 |
10.6379 |
0.5320 |
0.5188 |
37.6535 |
36.8969 |
27.4715 |
26.4881 |
0.5258 |
0.5300 |
| UKA-PRIMARY_1 |
9.9761 |
0.4705 |
10.2085 |
0.4873 |
0.4657 |
41.5185 |
40.3184 |
29.7032 |
28.2384 |
0.4908 |
0.5012 |
| UPC-PRIMARY-1 |
10.5986 |
0.5230 |
10.7301 |
0.5347 |
0.5220 |
36.9692 |
36.2986 |
27.1174 |
26.2783 |
0.5169 |
0.5252 |
| UPC-PRIMARY-2 |
10.5037 |
0.5149 |
10.6133 |
0.5253 |
0.5149 |
37.4231 |
36.8316 |
27.3202 |
26.5603 |
0.5126 |
0.5192 |
| USaar-Primary-Ptolemaios |
9.4704 |
0.4304 |
9.8477 |
0.4566 |
0.4303 |
56.3529 |
55.0291 |
30.9446 |
28.7267 |
0.4891 |
0.4960 |
| UW-PRIMARY-system1 |
10.5296 |
0.5261 |
10.7088 |
0.5401 |
0.5261 |
37.5744 |
36.7078 |
27.1758 |
26.0686 |
0.5250 |
0.5296 |
| UW-PRIMARY-system2 |
10.5513 |
0.5280 |
10.7284 |
0.5422 |
0.5280 |
37.5744 |
36.7078 |
27.4440 |
26.3333 |
0.5257 |
0.5302 |
| SYSTRAN** |
9.7174 |
0.4572 |
9.7369 |
0.4611 |
0.4573 |
42.0790 |
41.8418 |
30.7624 |
30.2913 |
0.4642 |
0.4595 |
*cs: case-sensitive
**translation with a commercial cd
| SYSTEM - Ranking |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_system1 |
2 |
1 |
2 |
2 |
3 |
2 |
2 |
2 |
2 |
6 |
5 |
| IBM_PRIMARY_system2 |
5 |
5 |
5 |
5 |
1 |
5 |
5 |
4 |
2 |
3 |
3 |
| IBM_PRIMARY_system3 |
4 |
3 |
4 |
4 |
5 |
4 |
4 |
5 |
5 |
5 |
6 |
| IBM_PRIMARY_system4 |
2 |
1 |
2 |
2 |
3 |
2 |
2 |
2 |
2 |
3 |
3 |
| IRST-PRIMARY |
8 |
9 |
6 |
9 |
10 |
10 |
8 |
10 |
7 |
11 |
8 |
| IRST-SECONDARY-1 |
14 |
13 |
11 |
11 |
15 |
14 |
14 |
15 |
14 |
13 |
11 |
| IRST-SECONDARY-2 |
15 |
14 |
12 |
12 |
13 |
15 |
15 |
14 |
11 |
12 |
12 |
| ROVER |
1 |
3 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
| RWTH-PRIMARY_1 |
6 |
6 |
7 |
8 |
6 |
7 |
7 |
6 |
8 |
2 |
2 |
| RWTH-PRIMARY_2 |
11 |
11 |
15 |
14 |
11 |
8 |
11 |
8 |
12 |
7 |
9 |
| UED-PRIMARY |
13 |
12 |
13 |
13 |
12 |
13 |
13 |
13 |
13 |
10 |
13 |
| UKA-PRIMARY_1 |
16 |
16 |
16 |
16 |
16 |
16 |
16 |
16 |
16 |
15 |
16 |
| UPC-PRIMARY-1 |
7 |
10 |
8 |
10 |
9 |
6 |
6 |
7 |
9 |
9 |
7 |
| UPC-PRIMARY-2 |
12 |
15 |
14 |
15 |
14 |
9 |
12 |
11 |
15 |
16 |
14 |
| USaar-Primary-Ptolemaios |
18 |
18 |
17 |
18 |
18 |
18 |
18 |
18 |
17 |
17 |
17 |
| UW-PRIMARY-system1 |
10 |
8 |
10 |
7 |
8 |
11 |
9 |
8 |
6 |
14 |
15 |
| UW-PRIMARY-system2 |
9 |
7 |
9 |
6 |
7 |
11 |
9 |
12 |
10 |
8 |
10 |
| SYSTRAN** |
17 |
17 |
18 |
17 |
17 |
17 |
17 |
17 |
18 |
18 |
18 |
*cs: case-sensitive
**translation with a commercial cd
EPPS / VERBATIM Task
| SYSTEM - Scoring |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_system1 |
10.8887 |
0.5508 |
10.9628 |
0.5590 |
0.5460 |
36.3516 |
35.6984 |
25.8358 |
25.1987 |
0.5147 |
0.5296 |
| IBM_PRIMARY_system2 |
10.9073 |
0.5524 |
10.9793 |
0.5602 |
0.5483 |
36.3066 |
35.6662 |
25.7425 |
25.1086 |
0.5158 |
0.5306 |
| IBM_PRIMARY_system3 |
10.8980 |
0.5518 |
10.9697 |
0.5596 |
0.5477 |
36.3420 |
35.7049 |
25.7779 |
25.1472 |
0.5153 |
0.5301 |
| IBM_PRIMARY_system4 |
10.9073 |
0.5524 |
10.9793 |
0.5602 |
0.5483 |
36.3066 |
35.6662 |
25.7425 |
25.1086 |
0.5158 |
0.5306 |
| IRST-PRIMARY ** |
10.5518 |
0.5208 |
10.7420 |
0.5355 |
0.5160 |
38.1761 |
37.3492 |
26.8398 |
25.8937 |
0.5057 |
0.5206 |
| IRST-SECONDARY-1 |
10.2303 |
0.5023 |
10.6718 |
0.5313 |
0.4958 |
40.2613 |
37.6870 |
29.1663 |
26.4890 |
0.5018 |
0.5187 |
| IRST-SECONDARY-2 |
10.2338 |
0.5024 |
10.6780 |
0.5319 |
0.4981 |
40.1519 |
37.5551 |
28.9700 |
26.2928 |
0.5047 |
0.52 |
| LIMSI-PRIMARY |
9.7619 |
0.4599 |
9.8583 |
0.4695 |
0.4547 |
42.4526 |
41.8348 |
30.0576 |
29.3979 |
0.4655 |
0.4811 |
| ROVER |
10.9912 |
0.5555 |
11.2207 |
0.5743 |
0.5489 |
35.6051 |
34.4435 |
25.3113 |
24.0596 |
0.5208 |
0.5377 |
| RWTH-PRIMARY_1 |
10.9431 |
0.5506 |
11.0360 |
0.5604 |
0.5453 |
36.4482 |
35.7853 |
25.8519 |
25.1762 |
0.5144 |
0.5308 |
| RWTH-PRIMARY_2 |
10.8110 |
0.5406 |
10.9165 |
0.5507 |
0.5366 |
37.0757 |
36.3999 |
26.4215 |
25.6749 |
0.511 |
0.5259 |
| RWTH-PRIMARY_3 |
10.7462 |
0.5312 |
10.8485 |
0.5411 |
0.5288 |
37.8640 |
37.2301 |
26.6049 |
25.8584 |
0.5082 |
0.5207 |
| UKA-PRIMARY_1 |
9.8504 |
0.4600 |
10.2556 |
0.4870 |
0.4548 |
43.1992 |
40.9917 |
30.3279 |
27.9210 |
0.477 |
0.4923 |
| UPC-PRIMARY |
10.4519 |
0.5200 |
10.6138 |
0.5318 |
0.5200 |
38.8422 |
38.0603 |
27.9467 |
27.0747 |
0.5098 |
0.5201 |
| USaar-Primary-Ptolemaios |
9.3324 |
0.4220 |
9.8715 |
0.4540 |
0.4202 |
61.4506 |
59.0372 |
31.7116 |
28.5163 |
0.472 |
0.4856 |
| UW-PRIMARY-system1 |
9.8480 |
0.4786 |
10.2508 |
0.5049 |
0.4786 |
42.7261 |
40.2870 |
30.9715 |
28.5388 |
0.5047 |
0.5118 |
| UW-PRIMARY-system2 |
9.8495 |
0.4800 |
10.2490 |
0.5061 |
0.4800 |
42.6232 |
40.2066 |
30.9940 |
28.5710 |
0.5058 |
0.5127 |
| SYSTRAN*** |
9.6760 |
0.4528 |
9.6847 |
0.4552 |
0.4528 |
43.1187 |
42.8581 |
30.5724 |
30.2410 |
0.4488 |
0.4496 |
*cs: case-sensitive
**submitted the 2006/03/29
***translation with a commercial cd
| SYSTEM - Ranking |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_system1 |
6 |
5 |
6 |
6 |
5 |
5 |
4 |
5 |
6 |
5 |
6 |
| IBM_PRIMARY_system2 |
3 |
2 |
3 |
3 |
2 |
2 |
2 |
2 |
2 |
2 |
3 |
| IBM_PRIMARY_system3 |
5 |
4 |
5 |
5 |
4 |
4 |
5 |
4 |
4 |
4 |
5 |
| IBM_PRIMARY_system4 |
3 |
2 |
3 |
3 |
2 |
2 |
2 |
2 |
2 |
2 |
3 |
| IRST-PRIMARY** |
9 |
9 |
9 |
9 |
10 |
9 |
9 |
9 |
9 |
11 |
9 |
| IRST-SECONDARY-1 |
12 |
12 |
11 |
12 |
12 |
12 |
11 |
12 |
11 |
14 |
12 |
| IRST-SECONDARY-2 |
11 |
11 |
10 |
10 |
11 |
11 |
10 |
11 |
10 |
12 |
11 |
| LIMSI-PRIMARY |
16 |
16 |
17 |
16 |
16 |
13 |
16 |
13 |
17 |
17 |
17 |
| ROVER |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
| RWTH-PRIMARY_1 |
2 |
6 |
2 |
2 |
6 |
6 |
6 |
6 |
5 |
6 |
2 |
| RWTH-PRIMARY_2 |
7 |
7 |
7 |
7 |
7 |
7 |
7 |
7 |
7 |
7 |
7 |
| RWTH-PRIMARY_3 |
8 |
8 |
8 |
8 |
8 |
8 |
8 |
8 |
8 |
9 |
8 |
| UKA-PRIMARY_1 |
13 |
15 |
13 |
15 |
15 |
17 |
15 |
14 |
13 |
15 |
15 |
| UPC-PRIMARY |
10 |
10 |
12 |
11 |
9 |
10 |
12 |
10 |
12 |
8 |
10 |
| USaar-Primary-Ptolemaios |
18 |
18 |
16 |
18 |
18 |
18 |
18 |
18 |
14 |
16 |
16 |
| UW-PRIMARY-system1 |
15 |
14 |
14 |
14 |
14 |
15 |
14 |
16 |
15 |
12 |
14 |
| UW-PRIMARY-system2 |
14 |
13 |
15 |
13 |
13 |
14 |
13 |
17 |
16 |
10 |
13 |
| SYSTRAN*** |
17 |
17 |
18 |
17 |
17 |
16 |
17 |
15 |
18 |
18 |
18 |
*cs: case-sensitive
**submitted the 2006/03/29
***translation with a commercial cd
EPPS / ASR Task
| SYSTEM - Scoring |
AS-WER |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_1 |
45.3694 |
9.6345 |
0.4265 |
9.7757 |
0.4374 |
0.4161 |
45.9827 |
45.1101 |
32.6543 |
31.6896 |
0.4953 |
0.5085 |
| IBM_PRIMARY_2 |
46.0576 |
9.5561 |
0.4228 |
9.6925 |
0.4337 |
0.4132 |
46.6310 |
45.7799 |
32.7834 |
31.8831 |
0.4974 |
0.5092 |
| IBM_PRIMARY_3 |
46.0863 |
9.5501 |
0.4229 |
9.6847 |
0.4336 |
0.4135 |
46.6402 |
45.8015 |
32.8079 |
31.9169 |
0.4969 |
0.5086 |
| IBM_PRIMARY_4 |
45.2753 |
9.6496 |
0.4284 |
9.7928 |
0.4394 |
0.4175 |
45.9028 |
45.0210 |
32.5529 |
31.5697 |
0.4966 |
0.5096 |
| IRST-PRIMARY** |
46.8480 |
9.2131 |
0.3793 |
9.3924 |
0.3918 |
0.3743 |
47.5927 |
46.7416 |
35.8159 |
34.8880 |
0.4852 |
0.5008 |
| IRST-SECONDARY-1 |
47.4435 |
8.7966 |
0.3583 |
9.2167 |
0.3826 |
0.3542 |
49.8295 |
47.2547 |
38.3384 |
35.6838 |
0.4776 |
0.496 |
| IRST-SECONDARY-2 |
47.2460 |
8.8712 |
0.3626 |
9.2874 |
0.3869 |
0.3579 |
49.6205 |
47.0765 |
38.0189 |
35.3888 |
0.4815 |
0.4981 |
| LIMSI-PRIMARY |
48.9893 |
8.7120 |
0.3660 |
8.8667 |
0.3773 |
0.3611 |
49.3717 |
48.4991 |
36.1201 |
35.1584 |
0.4339 |
0.4619 |
| RWTH-PRIMARY_1 |
46.0266 |
9.3815 |
0.3944 |
9.5396 |
0.4056 |
0.3901 |
46.5266 |
45.6386 |
35.5670 |
34.5931 |
0.4845 |
0.5039 |
| RWTH-PRIMARY_2 |
46.5479 |
9.2870 |
0.3855 |
9.4443 |
0.3968 |
0.3806 |
47.0550 |
46.1456 |
35.9634 |
35.0017 |
0.485 |
0.5028 |
| UKA-PRIMARY_1 |
53.4735 |
8.5306 |
0.3302 |
8.8595 |
0.3490 |
0.3275 |
52.2137 |
50.2258 |
39.2663 |
37.1494 |
0.4521 |
0.4697 |
| UPC-PRIMARY |
48.0449 |
9.1538 |
0.3833 |
9.3439 |
0.3956 |
0.3767 |
48.7510 |
47.8293 |
36.5840 |
35.5609 |
0.4851 |
0.4967 |
| SYSTRAN*** |
54.5987 |
8.5823 |
0.3379 |
8.7153 |
0.3474 |
0.3295 |
53.1631 |
52.5179 |
38.7962 |
38.0281 |
0.4401 |
0.4415 |
*cs: case-sensitive
**submitted the 2006/03/29
***translation with a commercial cd
| SYSTEM - Ranking |
AS-WER |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_1 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
3 |
2 |
| IBM_PRIMARY_2 |
4 |
3 |
4 |
3 |
3 |
4 |
4 |
4 |
3 |
3 |
2 |
3 |
| IBM_PRIMARY_3 |
5 |
4 |
3 |
4 |
4 |
3 |
5 |
5 |
4 |
4 |
4 |
4 |
| IBM_PRIMARY_4 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
| IRST-PRIMARY** |
7 |
7 |
8 |
7 |
8 |
8 |
7 |
7 |
6 |
6 |
7 |
7 |
| IRST-SECONDARY-1 |
9 |
10 |
11 |
10 |
10 |
11 |
11 |
9 |
11 |
11 |
9 |
9 |
| IRST-SECONDARY-2 |
8 |
9 |
10 |
9 |
9 |
10 |
10 |
8 |
10 |
9 |
8 |
8 |
| LIMSI-PRIMARY |
11 |
11 |
9 |
11 |
11 |
9 |
9 |
11 |
8 |
8 |
13 |
13 |
| RWTH-PRIMARY_1 |
3 |
5 |
5 |
5 |
5 |
5 |
3 |
3 |
5 |
5 |
5 |
5 |
| RWTH-PRIMARY_2 |
6 |
6 |
6 |
6 |
6 |
6 |
6 |
6 |
7 |
7 |
6 |
6 |
| UKA-PRIMARY_1 |
12 |
13 |
13 |
12 |
12 |
13 |
12 |
12 |
13 |
12 |
12 |
12 |
| UPC-PRIMARY |
10 |
8 |
7 |
8 |
7 |
7 |
8 |
10 |
9 |
10 |
11 |
11 |
| SYSTRAN*** |
13 |
12 |
12 |
13 |
13 |
12 |
13 |
13 |
12 |
13 |
10 |
10 |
*cs: case-sensitive
**submitted the 2006/03/29
***translation with a commercial cd
CORTES / FTE Task
| SYSTEM - Scoring |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_system1 |
9.2565 |
0.4208 |
9.3459 |
0.4309 |
0.4208 |
47.295 |
46.5407 |
34.0746 |
32.9433 |
0.4331 |
0.4323 |
| IBM_PRIMARY_system2 |
9.2348 |
0.4198 |
9.3225 |
0.4298 |
0.4198 |
47.517 |
46.7733 |
34.2368 |
33.116 |
0.4347 |
0.4326 |
| IBM_PRIMARY_system3 |
9.2135 |
0.4186 |
9.3009 |
0.4286 |
0.4186 |
47.5804 |
46.8368 |
34.3214 |
33.2006 |
0.4336 |
0.4315 |
| IBM_PRIMARY_system4 |
9.2348 |
0.4198 |
9.3225 |
0.4298 |
0.4198 |
47.517 |
46.7733 |
34.2368 |
33.116 |
0.4347 |
0.4326 |
| IRST-PRIMARY |
8.9627 |
0.3966 |
9.1933 |
0.4154 |
0.3966 |
49.4308 |
48.2008 |
35.5831 |
33.7892 |
0.4259 |
0.4302 |
| IRST-SECONDARY-1 |
8.9017 |
0.3902 |
9.1336 |
0.4089 |
0.3897 |
49.741 |
48.4862 |
35.8052 |
33.9971 |
0.4229 |
0.4281 |
| IRST-SECONDARY-2 |
8.8837 |
0.3901 |
9.1166 |
0.4089 |
0.3901 |
49.8009 |
48.5673 |
35.8439 |
34.0324 |
0.4233 |
0.4272 |
| ROVER |
9.2672 |
0.4195 |
9.5498 |
0.4411 |
0.4195 |
47.4359 |
45.9169 |
34.0218 |
31.96 |
0.4398 |
0.4409 |
| RWTH-PRIMARY_1 |
9.1268 |
0.4092 |
9.2122 |
0.4184 |
0.4093 |
48.8704 |
48.236 |
35.0509 |
34.0923 |
0.4392 |
0.4355 |
| RWTH-PRIMARY_2 |
9.0004 |
0.3987 |
9.0941 |
0.4086 |
0.3988 |
49.4061 |
48.7365 |
35.6325 |
34.6068 |
0.4322 |
0.4291 |
| UED-PRIMARY |
8.8454 |
0.3904 |
9.0998 |
0.4102 |
0.3904 |
49.9841 |
48.6907 |
35.9708 |
33.9865 |
0.4266 |
0.4249 |
| UKA-PRIMARY_1 |
8.4465 |
0.3517 |
8.7506 |
0.3722 |
0.351 |
52.9588 |
51.26 |
37.6696 |
35.273 |
0.4131 |
0.4163 |
| UPC-PRIMARY-1 |
9.0595 |
0.4037 |
9.216 |
0.4175 |
0.4038 |
48.2642 |
47.2914 |
34.9099 |
33.479 |
0.4297 |
0.4311 |
| UPC-PRIMARY-2 |
9.0105 |
0.3942 |
9.1531 |
0.4076 |
0.386 |
48.2008 |
47.3831 |
35.0721 |
33.8315 |
0.4119 |
0.4231 |
| USaar-Primary-Ptolemaios |
7.9067 |
0.311 |
8.3603 |
0.3397 |
0.3111 |
70.1265 |
68.5652 |
39.0477 |
35.992 |
0.403 |
0.4049 |
| UW-PRIMARY-system1 |
8.7567 |
0.383 |
9.0054 |
0.4012 |
0.383 |
50.7137 |
49.4378 |
36.422 |
34.4941 |
0.4222 |
0.4203 |
| UW-PRIMARY-system2 |
8.7815 |
0.389 |
9.0327 |
0.4079 |
0.389 |
50.3824 |
49.0819 |
36.2986 |
34.3425 |
0.4312 |
0.4286 |
| SYSTRAN** |
8.3203 |
0.3502 |
8.3277 |
0.3538 |
0.3501 |
52.7368 |
52.3455 |
38.667 |
38.0397 |
0.3751 |
0.3679 |
*cs: case-sensitive
**translation with a commercial cd
| SYSTEM - Ranking |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_system1 |
2 |
1 |
2 |
2 |
1 |
1 |
2 |
2 |
2 |
6 |
5 |
| IBM_PRIMARY_system2 |
3 |
2 |
3 |
3 |
2 |
3 |
3 |
3 |
3 |
3 |
3 |
| IBM_PRIMARY_system3 |
5 |
5 |
5 |
5 |
5 |
5 |
5 |
5 |
5 |
5 |
6 |
| IBM_PRIMARY_system4 |
3 |
2 |
3 |
3 |
2 |
3 |
3 |
3 |
3 |
3 |
3 |
| IRST-PRIMARY |
10 |
9 |
8 |
8 |
9 |
10 |
8 |
9 |
7 |
11 |
8 |
| IRST-SECONDARY-1 |
11 |
12 |
10 |
10 |
12 |
11 |
10 |
11 |
10 |
13 |
11 |
| IRST-SECONDARY-2 |
12 |
13 |
11 |
10 |
11 |
12 |
11 |
12 |
11 |
12 |
12 |
| ROVER |
1 |
4 |
1 |
1 |
4 |
2 |
1 |
1 |
1 |
1 |
1 |
| RWTH-PRIMARY_1 |
6 |
6 |
7 |
6 |
6 |
8 |
9 |
7 |
12 |
2 |
2 |
| RWTH-PRIMARY_2 |
9 |
8 |
13 |
12 |
8 |
9 |
13 |
10 |
15 |
7 |
9 |
| UED-PRIMARY |
13 |
11 |
12 |
9 |
10 |
13 |
12 |
13 |
9 |
10 |
13 |
| UKA-PRIMARY_1 |
16 |
16 |
16 |
16 |
16 |
17 |
16 |
16 |
16 |
15 |
16 |
| UPC-PRIMARY-1 |
7 |
7 |
6 |
7 |
7 |
7 |
6 |
6 |
6 |
9 |
7 |
| UPC-PRIMARY-2 |
8 |
10 |
9 |
14 |
14 |
6 |
7 |
8 |
8 |
16 |
14 |
| USaar-Primary-Ptolemaios |
18 |
18 |
17 |
18 |
18 |
18 |
18 |
18 |
17 |
17 |
17 |
| UW-PRIMARY-system1 |
15 |
15 |
15 |
15 |
15 |
15 |
15 |
15 |
14 |
14 |
15 |
| UW-PRIMARY-system2 |
14 |
14 |
14 |
13 |
13 |
14 |
14 |
14 |
13 |
8 |
10 |
| SYSTRAN** |
17 |
17 |
18 |
17 |
17 |
16 |
17 |
17 |
18 |
18 |
18 |
*cs: case-sensitive
**translation with a commercial cd
CORTES / VERBATIM Task
| SYSTEM - Scoring |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_system1 |
10.198 |
0.5014 |
10.2841 |
0.51 |
0.5014 |
38.5393 |
37.7598 |
28.1115 |
27.2909 |
0.5149 |
0.5216 |
| IBM_PRIMARY_system2 |
10.198 |
0.5014 |
10.2841 |
0.51 |
0.5014 |
38.5393 |
37.7598 |
28.1115 |
27.2909 |
0.5149 |
0.5216 |
| IBM_PRIMARY_system3 |
10.1774 |
0.4997 |
10.2633 |
0.5084 |
0.4997 |
38.6246 |
37.845 |
28.1936 |
27.373 |
0.5135 |
0.5203 |
| IBM_PRIMARY_system4 |
10.2067 |
0.5017 |
10.2933 |
0.5105 |
0.5017 |
38.4889 |
37.7061 |
28.0516 |
27.2278 |
0.5145 |
0.5216 |
| IRST-PRIMARY** |
9.6768 |
0.457 |
10.0015 |
0.4797 |
0.4545 |
41.6607 |
40.1805 |
30.4471 |
28.7554 |
0.4879 |
0.5014 |
| IRST-SECONDARY-1 |
9.4295 |
0.4417 |
9.9331 |
0.4735 |
0.4376 |
43.4787 |
40.6413 |
32.2176 |
29.2067 |
0.4817 |
0.4965 |
| IRST-SECONDARY-2 |
9.4258 |
0.4418 |
9.9301 |
0.4738 |
0.4386 |
43.4376 |
40.5877 |
32.0882 |
29.0742 |
0.4837 |
0.4976 |
| LIMSI-PRIMARY |
8.8798 |
0.3922 |
9.004 |
0.403 |
0.3872 |
46.5085 |
45.5838 |
33.4359 |
32.4859 |
0.4332 |
0.4495 |
| ROVER |
10.2043 |
0.4965 |
10.5951 |
0.5239 |
0.4935 |
38.0123 |
36.0997 |
28.2252 |
26.1674 |
0.5105 |
0.5254 |
| RWTH-PRIMARY_1 |
10.2457 |
0.4988 |
10.3764 |
0.5107 |
0.4933 |
38.9844 |
38.1322 |
28.9416 |
28.0232 |
0.4996 |
0.5176 |
| RWTH-PRIMARY_2 |
10.1648 |
0.4916 |
10.3063 |
0.5049 |
0.4882 |
38.6814 |
37.7566 |
28.6039 |
27.6507 |
0.5024 |
0.5159 |
| RWTH-PRIMARY_3 |
10.0239 |
0.4786 |
10.1717 |
0.4927 |
0.4786 |
39.6756 |
38.6877 |
28.7996 |
27.7959 |
0.5022 |
0.5114 |
| UKA-PRIMARY_1 |
9.1095 |
0.4045 |
9.6009 |
0.4357 |
0.4027 |
46.4675 |
43.7248 |
32.7321 |
29.6928 |
0.4635 |
0.4756 |
| UPC-PRIMARY |
9.9062 |
0.4728 |
10.0499 |
0.4846 |
0.4706 |
40.3446 |
39.4862 |
29.7306 |
28.8532 |
0.4822 |
0.4961 |
| USaar-Primary-Ptolemaios |
8.1796 |
0.3282 |
8.766 |
0.3589 |
0.3283 |
75.6269 |
73.4144 |
35.3138 |
31.6275 |
0.4319 |
0.4427 |
| UW-PRIMARY-system1 |
9.1093 |
0.4213 |
9.5709 |
0.4496 |
0.4213 |
45.9815 |
43.3619 |
33.5653 |
30.6301 |
0.4803 |
0.4886 |
| UW-PRIMARY-system2 |
9.1146 |
0.4218 |
9.5821 |
0.4507 |
0.4218 |
45.9436 |
43.2672 |
33.5969 |
30.6459 |
0.4808 |
0.4892 |
| SYSTRAN*** |
9.2589 |
0.422 |
9.2691 |
0.4251 |
0.4222 |
44.0121 |
43.7122 |
31.8137 |
31.4791 |
0.4476 |
0.4494 |
*cs: case-sensitive
**submitted the 2006/03/29
***translation with a commercial cd
| SYSTEM - Ranking |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_system1 |
4 |
2 |
5 |
4 |
2 |
3 |
4 |
2 |
3 |
1 |
2 |
| IBM_PRIMARY_system2 |
4 |
2 |
5 |
4 |
2 |
3 |
4 |
2 |
3 |
1 |
2 |
| IBM_PRIMARY_system3 |
6 |
4 |
7 |
6 |
4 |
5 |
6 |
4 |
5 |
4 |
5 |
| IBM_PRIMARY_system4 |
2 |
1 |
4 |
3 |
1 |
2 |
2 |
1 |
2 |
3 |
2 |
| IRST-PRIMARY ** |
10 |
10 |
10 |
10 |
10 |
10 |
10 |
10 |
9 |
9 |
9 |
| IRST-SECONDARY-1 |
11 |
12 |
11 |
12 |
12 |
12 |
12 |
13 |
12 |
12 |
11 |
| IRST-SECONDARY-2 |
12 |
11 |
12 |
11 |
11 |
11 |
11 |
12 |
11 |
10 |
10 |
| LIMSI-PRIMARY |
17 |
17 |
17 |
17 |
17 |
17 |
17 |
15 |
18 |
17 |
16 |
| ROVER |
3 |
6 |
1 |
1 |
5 |
1 |
1 |
5 |
1 |
5 |
1 |
| RWTH-PRIMARY_1 |
1 |
5 |
2 |
2 |
6 |
7 |
7 |
8 |
8 |
8 |
6 |
| RWTH-PRIMARY_2 |
7 |
7 |
3 |
7 |
7 |
6 |
3 |
6 |
6 |
6 |
7 |
| RWTH-PRIMARY_3 |
8 |
8 |
8 |
8 |
8 |
8 |
8 |
7 |
7 |
7 |
8 |
| UKA-PRIMARY_1 |
15 |
16 |
13 |
15 |
16 |
16 |
16 |
14 |
13 |
15 |
15 |
| UPC-PRIMARY |
9 |
9 |
9 |
9 |
9 |
9 |
9 |
9 |
10 |
11 |
12 |
| USaar-Primary-Ptolemaios |
18 |
18 |
18 |
18 |
18 |
18 |
18 |
18 |
17 |
18 |
18 |
| UW-PRIMARY-system1 |
16 |
15 |
15 |
14 |
15 |
15 |
14 |
16 |
14 |
14 |
14 |
| UW-PRIMARY-system2 |
14 |
14 |
14 |
13 |
14 |
14 |
13 |
17 |
15 |
13 |
13 |
| SYSTRAN*** |
13 |
13 |
16 |
16 |
13 |
13 |
15 |
11 |
16 |
16 |
17 |
*cs: case-sensitive
**submitted the 2006/03/29
***translation with a commercial cd
CORTES / ASR Task
| SYSTEM - Scoring |
AS-WER |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_1 |
49.4953 |
8.7098 |
0.3606 |
8.9207 |
0.373 |
0.3503 |
51.5452 |
50.2448 |
36.8414 |
35.4623 |
0.4713 |
0.4856 |
| IBM_PRIMARY_2 |
50.0155 |
8.6855 |
0.3573 |
8.9022 |
0.3705 |
0.3487 |
52.2504 |
50.9312 |
36.8004 |
35.3804 |
0.4724 |
0.4845 |
| IBM_PRIMARY_3 |
50.1642 |
8.6673 |
0.3562 |
8.8826 |
0.3693 |
0.347 |
52.389 |
51.0823 |
36.9547 |
35.5442 |
0.4709 |
0.4832 |
| IBM_PRIMARY_4 |
49.6247 |
8.7081 |
0.3621 |
8.9187 |
0.3745 |
0.3516 |
51.7341 |
50.4432 |
36.8603 |
35.4591 |
0.4744 |
0.4869 |
| IRST-PRIMARY** |
51.2463 |
8.0848 |
0.3053 |
8.3642 |
0.3207 |
0.3026 |
53.4784 |
52.008 |
41.4005 |
39.7538 |
0.4508 |
0.4719 |
| IRST-SECONDARY-1 |
51.4206 |
7.7438 |
0.2909 |
8.2584 |
0.3175 |
0.2891 |
55.2825 |
52.2001 |
43.362 |
40.0655 |
0.446 |
0.4698 |
| IRST-SECONDARY-2 |
51.2938 |
7.7858 |
0.2925 |
8.2974 |
0.3192 |
0.2897 |
55.2384 |
52.1497 |
43.1857 |
39.9364 |
0.4496 |
0.4716 |
| LIMSI-PRIMARY |
54.4732 |
7.5794 |
0.2839 |
7.7369 |
0.2939 |
0.2796 |
55.8618 |
54.848 |
40.2922 |
39.2217 |
0.39 |
0.4232 |
| RWTH-PRIMARY_1 |
49.4775 |
8.3381 |
0.327 |
8.5605 |
0.3394 |
0.3246 |
51.7309 |
50.3644 |
40.755 |
39.3161 |
0.4559 |
0.481 |
| RWTH-PRIMARY_2 |
50.1909 |
8.2783 |
0.3196 |
8.4993 |
0.3321 |
0.3158 |
52.4079 |
51.0666 |
41.1927 |
39.7727 |
0.4553 |
0.4776 |
| UKA-PRIMARY_1 |
55.7988 |
7.6327 |
0.2712 |
8.0504 |
0.2915 |
0.2672 |
57.4991 |
55.0653 |
43.6328 |
40.925 |
0.4255 |
0.4458 |
| UPC-PRIMARY |
51.1048 |
8.1449 |
0.3119 |
8.4083 |
0.3251 |
0.3077 |
53.3304 |
51.8758 |
41.4918 |
39.8986 |
0.4412 |
0.4615 |
| SYSTRAN*** |
56.3094 |
7.8623 |
0.2848 |
8.0369 |
0.2942 |
0.2772 |
57.4613 |
56.3656 |
42.8268 |
41.602 |
0.4177 |
0.4242 |
*cs: case-sensitive
**submitted the 2006/03/29
***translation with a commercial cd
| SYSTEM - Ranking |
AS-WER |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| IBM_PRIMARY_1 |
2 |
1 |
2 |
1 |
2 |
2 |
1 |
1 |
2 |
3 |
3 |
2 |
| IBM_PRIMARY_2 |
4 |
3 |
3 |
3 |
3 |
3 |
4 |
4 |
1 |
1 |
2 |
3 |
| IBM_PRIMARY_3 |
5 |
4 |
4 |
4 |
4 |
4 |
5 |
6 |
4 |
4 |
4 |
4 |
| IBM_PRIMARY_4 |
3 |
2 |
1 |
2 |
1 |
1 |
3 |
3 |
3 |
2 |
1 |
1 |
| IRST-PRIMARY** |
8 |
8 |
8 |
8 |
8 |
8 |
8 |
8 |
8 |
7 |
7 |
7 |
| IRST-SECONDARY-1 |
10 |
11 |
10 |
10 |
10 |
10 |
10 |
10 |
12 |
11 |
9 |
9 |
| IRST-SECONDARY-2 |
9 |
10 |
9 |
9 |
9 |
9 |
9 |
9 |
11 |
10 |
8 |
8 |
| LIMSI-PRIMARY |
11 |
13 |
12 |
13 |
12 |
11 |
11 |
11 |
5 |
5 |
13 |
13 |
| RWTH-PRIMARY_1 |
1 |
5 |
5 |
5 |
5 |
5 |
2 |
2 |
6 |
6 |
5 |
5 |
| RWTH-PRIMARY_2 |
6 |
6 |
6 |
6 |
6 |
6 |
6 |
5 |
7 |
8 |
6 |
6 |
| UKA-PRIMARY_1 |
12 |
12 |
13 |
11 |
13 |
13 |
13 |
12 |
13 |
12 |
11 |
11 |
| UPC-PRIMARY |
7 |
7 |
7 |
7 |
7 |
7 |
7 |
7 |
9 |
9 |
10 |
10 |
| SYSTRAN*** |
13 |
9 |
11 |
12 |
11 |
12 |
12 |
13 |
10 |
13 |
12 |
12 |
*cs: case-sensitive
**submitted the 2006/03/29
***translation with a commercial cd
Last update: 29th August 2006