SLT Results for the Chinese to English direction
(in red the best primary system for each metric)
VERBATIM Task
| SYSTEM - Scoring |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| ICT-PRIMARY-BRUIN1 |
5.9553 |
0.1359 |
6.2095 |
0.1463 |
0.1309 |
80.9258 |
79.8568 |
59.0836 |
56.6477 |
0.2743 |
0.2848 |
| ICT-PRIMARY-BRUIN2 |
5.8304 |
0.1243 |
6.0721 |
0.1339 |
0.1195 |
78.7361 |
77.6898 |
59.0318 |
56.7449 |
0.2617 |
0.2804 |
| ICT-PRIMARY-Confucius1 |
6.0294 |
0.1370 |
6.2765 |
0.1467 |
0.1320 |
78.6810 |
77.6606 |
58.0633 |
55.8088 |
0.2700 |
0.2875 |
| ICT-PRIMARY-Confucius2 |
5.8939 |
0.1298 |
6.1338 |
0.1395 |
0.1298 |
83.1706 |
82.1502 |
60.5316 |
58.1799 |
0.2866 |
0.2900 |
| ICT-PRIMARY-LYNX1 |
5.7246 |
0.1263 |
5.9751 |
0.1364 |
0.1207 |
81.6093 |
80.2617 |
60.7194 |
58.5718 |
0.2772 |
0.2909 |
| ICT-PRIMARY-LYNX2 |
5.5098 |
0.1098 |
5.7891 |
0.1203 |
0.1054 |
80.7314 |
79.3742 |
60.9494 |
58.5686 |
0.2426 |
0.2608 |
| IRST-PRIMARY |
6.0136 |
0.1404 |
6.3611 |
0.1534 |
0.1349 |
79.7564 |
78.3830 |
59.4173 |
56.4922 |
0.2788 |
0.2912 |
| IRST-SECONDARY-1 |
5.9993 |
0.1392 |
6.3458 |
0.1521 |
0.1342 |
80.0479 |
78.6810 |
59.6634 |
56.7319 |
0.2789 |
0.2911 |
| IRST-SECONDARY-2 |
5.6126 |
0.1187 |
5.9658 |
0.1305 |
0.1152 |
77.9716 |
76.5820 |
59.9193 |
57.0235 |
0.2489 |
0.2704 |
| IRST-SECONDARY-3 |
5.9619 |
0.1410 |
6.2502 |
0.1529 |
0.1411 |
82.6620 |
81.4279 |
61.3640 |
58.7079 |
0.2907 |
0.2932 |
| NLPR-SECONDARY |
4.3532 |
0.0730 |
4.6274 |
0.0821 |
0.0730 |
102.9234 |
101.7055 |
79.8212 |
76.6467 |
0.2618 |
0.2622 |
| NRC-PRIMARY |
5.4932 |
0.1224 |
5.8122 |
0.1359 |
0.1225 |
84.5019 |
83.2937 |
63.0711 |
60.3145 |
0.2885 |
0.2946 |
| NRC-SECONDARY |
5.7967 |
0.1276 |
6.1303 |
0.1415 |
0.1276 |
84.2266 |
82.8564 |
61.6718 |
58.6593 |
0.2983 |
0.3023 |
| RWTH-PRIMARY_1 |
6.4465 |
0.1607 |
6.7456 |
0.1750 |
0.1532 |
78.0785 |
76.8800 |
56.3432 |
53.8101 |
0.2897 |
0.3023 |
| RWTH-PRIMARY_2 |
6.4108 |
0.1602 |
6.6999 |
0.1738 |
0.1524 |
77.8323 |
76.6305 |
56.4663 |
53.9592 |
0.2883 |
0.3018 |
| RWTH-PRIMARY_3 |
6.0676 |
0.1532 |
6.3578 |
0.1669 |
0.1476 |
76.7925 |
75.6296 |
56.7384 |
54.2442 |
0.2805 |
0.301 |
| UKA-PRIMARY_1 |
5.5139 |
0.1081 |
5.791 |
0.1174 |
0.1030 |
82.2215 |
81.0813 |
61.7204 |
59.0804 |
0.2538 |
0.2689 |
| UKA-PRIMARY_2 |
5.4902 |
0.1135 |
5.7594 |
0.1248 |
0.1085 |
81.2821 |
80.1192 |
61.9957 |
59.5468 |
0.2551 |
0.2725 |
| UKA-PRIMARY_3 |
5.4090 |
0.1035 |
5.6755 |
0.1123 |
0.1015 |
84.4954 |
83.3714 |
63.5117 |
60.8911 |
0.2538 |
0.2689 |
| UKA-PRIMARY_4 |
5.1082 |
0.0985 |
5.3746 |
0.1088 |
0.0985 |
89.9276 |
88.6092 |
68.1729 |
65.4131 |
0.2740 |
0.2694 |
| SYSTRAN** |
4.2782 |
0.0653 |
4.5674 |
0.0747 |
0.0653 |
95.3695 |
93.7402 |
74.7745 |
71.9045 |
0.2428 |
0.2300 |
*cs: case-sensitive
**translation with a commercial cd
| SYSTEM - Ranking |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| ICT-PRIMARY-BRUIN1 |
8 |
8 |
8 |
8 |
8 |
10 |
10 |
6 |
6 |
11 |
12 |
| ICT-PRIMARY-BRUIN2 |
10 |
12 |
11 |
13 |
13 |
6 |
6 |
5 |
8 |
15 |
13 |
| ICT-PRIMARY-Confucius1 |
4 |
7 |
6 |
7 |
7 |
5 |
5 |
4 |
4 |
13 |
11 |
| ICT-PRIMARY-Confucius2 |
9 |
9 |
9 |
10 |
9 |
15 |
15 |
10 |
10 |
6 |
10 |
| ICT-PRIMARY-LYNX1 |
12 |
11 |
12 |
11 |
12 |
12 |
12 |
11 |
12 |
10 |
9 |
| ICT-PRIMARY-LYNX2 |
15 |
16 |
16 |
16 |
16 |
9 |
9 |
12 |
11 |
21 |
20 |
| IRST-PRIMARY |
5 |
5 |
3 |
4 |
5 |
7 |
7 |
7 |
5 |
9 |
7 |
| IRST-SECONDARY-1 |
6 |
6 |
5 |
6 |
6 |
8 |
8 |
8 |
7 |
8 |
8 |
| IRST-SECONDARY-2 |
13 |
14 |
13 |
14 |
14 |
3 |
2 |
9 |
9 |
19 |
15 |
| IRST-SECONDARY-3 |
7 |
4 |
7 |
5 |
4 |
14 |
14 |
13 |
14 |
2 |
6 |
| NLPR-SECONDARY |
20 |
20 |
20 |
20 |
20 |
21 |
21 |
21 |
21 |
14 |
19 |
| NRC-PRIMARY |
16 |
13 |
14 |
12 |
11 |
18 |
17 |
17 |
17 |
4 |
5 |
| NRC-SECONDARY |
11 |
10 |
10 |
9 |
10 |
16 |
16 |
14 |
13 |
1 |
1 |
| RWTH-PRIMARY_1 |
1 |
1 |
1 |
1 |
1 |
4 |
4 |
1 |
1 |
3 |
1 |
| RWTH-PRIMARY_2 |
2 |
2 |
2 |
2 |
2 |
2 |
3 |
2 |
2 |
5 |
3 |
| RWTH-PRIMARY_3 |
3 |
3 |
4 |
3 |
3 |
1 |
1 |
3 |
3 |
7 |
4 |
| UKA-PRIMARY_1 |
14 |
17 |
15 |
17 |
17 |
13 |
13 |
15 |
15 |
17 |
17 |
| UKA-PRIMARY_2 |
17 |
15 |
17 |
15 |
15 |
11 |
11 |
16 |
16 |
16 |
14 |
| UKA-PRIMARY_3 |
18 |
18 |
18 |
18 |
18 |
17 |
18 |
18 |
18 |
17 |
17 |
| UKA-PRIMARY_4 |
19 |
19 |
19 |
19 |
19 |
19 |
19 |
19 |
19 |
12 |
16 |
| SYSTRAN** |
21 |
21 |
21 |
21 |
21 |
20 |
20 |
20 |
20 |
20 |
21 |
*cs: case-sensitive
**translation with a commercial cd
ASR Task
| SYSTEM - Scoring |
AS-WER |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| ICT-PRIMARY-BRUIN |
86.8727 |
4.4265 |
0.0931 |
4.6563 |
0.1003 |
0.0899 |
79.7551 |
78.5628 |
64.7209 |
62.5376 |
0.254 |
0.2713 |
| ICT-PRIMARY-Confucius1 |
83.3082 |
4.8978 |
0.1086 |
5.1268 |
0.1184 |
0.1046 |
77.7859 |
76.4625 |
62.4576 |
60.4149 |
0.2615 |
0.2800 |
| ICT-PRIMARY-Confucius2 |
84.7947 |
4.8080 |
0.1028 |
5.0640 |
0.1124 |
0.1006 |
78.9400 |
77.5014 |
63.1066 |
60.8401 |
0.2678 |
0.2840 |
| IRST-PRIMARY |
83.5759 |
4.9208 |
0.1107 |
5.2390 |
0.1234 |
0.1083 |
78.8025 |
77.0155 |
63.1897 |
60.4277 |
0.2714 |
0.2856 |
| IRST-SECONDARY-1 |
83.7788 |
4.9524 |
0.1112 |
5.2745 |
0.1242 |
0.1088 |
78.9304 |
77.1466 |
63.2217 |
60.4597 |
0.2705 |
0.2852 |
| IRST-SECONDARY-2 |
82.7052 |
4.4758 |
0.0942 |
4.7877 |
0.1059 |
0.0904 |
76.6191 |
74.9121 |
62.9435 |
60.2871 |
0.2417 |
0.2653 |
| IRST-SECONDARY-3 |
85.0429 |
4.6952 |
0.0990 |
5.0160 |
0.1121 |
0.0980 |
79.6976 |
77.9650 |
64.5323 |
61.8982 |
0.2655 |
0.2788 |
| NLPR-SECONDARY |
90.4877 |
4.0937 |
0.0674 |
4.3910 |
0.0765 |
0.0674 |
87.2770 |
85.6435 |
71.2742 |
68.4259 |
0.2515 |
0.2583 |
| RWTH-PRIMARY_1 |
83.0153 |
5.1680 |
0.1239 |
5.4361 |
0.1350 |
0.1209 |
77.9873 |
76.4849 |
61.9845 |
59.6605 |
0.2788 |
0.2941 |
| RWTH-PRIMARY_2 |
82.6381 |
5.1628 |
0.1249 |
5.4254 |
0.1360 |
0.1209 |
77.5909 |
76.1077 |
61.6233 |
59.3376 |
0.2778 |
0.2941 |
| RWTH-PRIMARY_3 |
81.1004 |
4.9256 |
0.1259 |
5.1637 |
0.1369 |
0.1200 |
75.4811 |
74.0266 |
60.3990 |
58.2348 |
0.2698 |
0.2927 |
| UKA-PRIMARY_1 |
88.2787 |
4.5916 |
0.0846 |
4.8424 |
0.0935 |
0.0846 |
82.9167 |
81.4718 |
66.8851 |
64.6218 |
0.2535 |
0.2671 |
| UKA-PRIMARY_2 |
86.6337 |
4.3821 |
0.0862 |
4.5992 |
0.0942 |
0.0848 |
80.5895 |
79.2341 |
65.7822 |
63.6884 |
0.2438 |
0.2650 |
| SYSTRAN*** |
95.2022 |
3.7299 |
0.0579 |
3.9897 |
0.0666 |
0.0579 |
93.9294 |
92.2288 |
78.4285 |
75.7496 |
0.2402 |
0.2310 |
*cs: case-sensitive
**need to be manually corrected
***translation with a commercial cd
| SYSTEM - Ranking |
AS-WER |
NIST cs* |
BLEU cs* |
NIST |
BLEU |
IBM |
mWER cs* |
mWER |
mPER cs* |
mPER |
WNM / RECALL |
WNM / F-measure |
| ICT-PRIMARY-BRUIN |
11 |
11 |
10 |
11 |
10 |
10 |
10 |
10 |
10 |
10 |
9 |
9 |
| ICT-PRIMARY-Confucius1 |
5 |
6 |
6 |
6 |
6 |
6 |
4 |
4 |
4 |
5 |
8 |
7 |
| ICT-PRIMARY-Confucius2 |
8 |
7 |
7 |
7 |
7 |
7 |
8 |
8 |
6 |
8 |
6 |
6 |
| IRST-PRIMARY |
6 |
5 |
5 |
4 |
5 |
5 |
6 |
6 |
7 |
6 |
3 |
4 |
| IRST-SECONDARY-1 |
7 |
3 |
4 |
3 |
4 |
4 |
7 |
7 |
8 |
7 |
4 |
5 |
| IRST-SECONDARY-2 |
3 |
10 |
9 |
10 |
9 |
9 |
2 |
2 |
5 |
4 |
13 |
11 |
| IRST-SECONDARY-3 |
9 |
8 |
8 |
8 |
8 |
8 |
9 |
9 |
9 |
9 |
7 |
8 |
| NLPR-SECONDARY |
13 |
13 |
13 |
13 |
13 |
13 |
13 |
13 |
13 |
13 |
11 |
13 |
| RWTH-PRIMARY_1 |
4 |
1 |
3 |
1 |
3 |
1 |
5 |
5 |
3 |
3 |
1 |
1 |
| RWTH-PRIMARY_2 |
2 |
2 |
2 |
2 |
2 |
1 |
3 |
3 |
2 |
2 |
2 |
1 |
| RWTH-PRIMARY_3 |
1 |
4 |
1 |
5 |
1 |
3 |
1 |
1 |
1 |
1 |
5 |
3 |
| UKA-PRIMARY_1 |
12 |
9 |
12 |
9 |
12 |
12 |
12 |
12 |
12 |
12 |
10 |
10 |
| UKA-PRIMARY_2 |
10 |
12 |
11 |
12 |
11 |
11 |
11 |
11 |
11 |
11 |
12 |
12 |
| SYSTRAN*** |
14 |
14 |
14 |
14 |
14 |
14 |
14 |
14 |
14 |
14 |
14 |
14 |
*cs: case-sensitive
***translation with a commercial cd
Last update: 29th August 2006