Méthodologie d’Evaluation automatique de la compréhension hors et en contexte du DIAlogue
(Automatic evaluation of man-machine dialogue systems)

Presentation

The aim of the MEDIA evaluation campaign is to test an automatic evaluation methodology for man-machine dialogue systems. The evaluation methodology is based on a paradigm that uses test sets taken from a corpus of real-world dialogues, a semantic representation of dialogue and common evaluation metrics. This protocol is designed to test the capacity of dialogue systems, both taking into account and not taking into account, the context of the dialogue.

In order to validate the evaluation protocol and the semantic representations, an evaluation campaign will take place where each partner in the project tests their system. The task chosen is hotel room reservation, with touristic information as an additional point of entry into the dialogue.

The final Media Workshop took place at the Sainte-Marthe university in Avignon, France, on July 6-7 2006.

Links

MEDIA restricted area available here

For more information (in French), please visit Technolangue.Net.