Affiliations: Carnegie Mellon University, Language Technologies and
Human-Computer Interaction, 5000 Forbes Avenue, Pittsburgh PA, 15213, USA.
E-mail: cprose@cs.cmu.edu; http://www.cs.cmu.edu/~cprose | University of Pittsburgh, Learning Research and
Development Center, 3939 O'Hara Street, Pittsburgh PA, 15260, USA.
E-mail: vanlehn@pitt.edu; http://www.pitt.edu/~vanlehn
Abstract: In this paper, we explore the problem of selecting appropriate
interventions for students based on an analysis of their interactions with a
tutoring system. In the context of the WHY2 conceptual physics tutoring system,
we describe CarmelTC, a hybrid symbolic/statistical approach for analysing
conceptual physics explanations in order to determine which Knowledge
Construction Dialogues (KCDs) students need for the purpose of encouraging them
to include important points that are missing. We briefly describe our tutoring
approach. We then present a model that demonstrates a general problem with
selecting interventions based on an analysis of student performance in
circumstances where there is uncertainty with the interpretation, such as with
speech or text based natural language input, complex and error prone
mathematical or other formal language input, graphical input (i.e., diagrams,
etc.), or gestures. In particular, when student performance completeness is
high, intervention selection accuracy is more sensitive to analysis accuracy,
and increasingly so as performance completeness increases. In light of this
model, we have evaluated our CarmelTC approach and have demonstrated that it
performs favourably in comparison with the widely used LSA approach, a Naive
Bayes approach, and finally a purely symbolic approach.
Keywords: Tutorial dialogue, language understanding, evaluation
