A corpus-based approach for cooperative response generation in a dialog system

This paper presents a corpus-based approach for cooperative response generation in a spoken dialog system for the Hong Kong tourism domain. A corpus with 3874 requests and responses is collected using Wizard-of-Oz framework. The corpus then undergoes a regularization process that simplifies the interactions to ease subsequent modeling. A semi-automatic process is developed to annotate each utterance in the dialog turns in terms of their key concepts (KC), task goal (TG) and dialog acts (DA). TG and DA characterize the informational goal and communicative goal of the utterance respectively. The annotation procedure is integrated with a dialog modeling heuristic and a discourse inheritance strategy to generate a semantic abstraction (SA), in the form of {TG, DA, KC}, for each user request and system response in the dialog. Semantic transitions, i.e. {TG, DA, KC}(user) -> {TG, DA, KC}(system), may hence be directly derived from the corpus as rules for response message planning. Related verbalization methods may also be derived from the corpus and used as templates for response message realization. All the rules and templates are stored externally in a human-readable text file which brings the advantage of easy extensibility of the system. Evaluation of this corpus based approach shows that 83% of the generated responses are coherent with the user's request and qualitative rating achieves a score of 4.0 on a five-point Likert scale.