Affiliations: Department of Computer Science, University of Illinois
at Chicago, Chicago, IL, 60302, USA. {bdieugen,dfossa1}@uic.edu | Department of Computer Science, SUNY Potsdam, Potsdam
NY 13676, USA. hallersm@potsdam.edu | Albert A. Webb Associates, Riverside, CA 92506, USA.
danyu79@gmail.com | Department of Math and Computer Science, Valparaiso
University, Valparaiso, IN 46383, USA. michael.glass@valpo.edu
Abstract: To investigate whether more concise Natural Language feedback
improves learning, we developed two Natural Language generators (DIAG-NLP1 and
DIAG-NLP2), to provide feedback in an Intelligent Tutoring System that teaches
troubleshooting. We systematically evaluated them in a three way comparison
that included the original system, which generates overly repetitive feedback.
We found that DIAG-NLP2, the generator which intuitively produces the best,
corpus-based language, does engender the most learning. Distinguishing features
of the more effective feedback are: it obeys Grice's maxim of
brevity, it is more directive and uses a specific type of referring
expressions. Interestingly, simpler ways of restructuring the original
repetitive feedback as done in DIAG-NLP1, such as exploiting the hierarchical
structure of the domain, were not effective. Since the design of interfaces to
Intelligent Tutoring Systems often includes verbal feedback, we suggest that:
if the number of different contexts in which verbal feedback is provided is
high, such feedback should be based on corpus studies, and generated by
techniques more sophisticated than template filling.
Keywords: Intelligent tutoring systems, natural language interfaces, corpus studies, feedback generation
