Assessing debate strategies via computational agents

1.Introduction

Computer-based learning, aka e-learning or technology-enhanced learning, is becoming increasingly important in education, in general, and in higher education, in particular. And it does seem to have many advantages. For example, students may learn at their own pace and convenience, remotely from the university campus if necessary. A possible danger with e-learning, however, is that the educational interactions may become unduly didactic (Moore 2000). Computer-based learning systems, that is, tend to adopt what Freire (2000) calls the banking concept of education – students are (partially) empty vessels into which suitable knowledge is deposited. This concern is very real today; the use, for example, of multiple choice quizzes within webCT exemplifies the banking concept: there is a single right answer to each question, the aim is to get students to learn the right answers. While this approach may be valid in some domains, its approach is limited in attempts to encourage critical thinking and reflection, the vital components of higher education.

We argue (e.g. Moore 2000; Naim, Moore, Yuan, Dixon and Grierson 2009) that there are two possible approaches to addressing this problem of untoward didacticism. One is to allow multiple participants in the learning interactions, so that learners are able to use the environment to communicate with each other and their tutors. A different approach is to have the computer itself be a participant in the learning interaction. If the computer is to engage in dialogue with students, it needs a model of dialogue, and such a model is potentially provided by computational dialectics. Computational dialectics is a maturing strand of research that is focused on computational utilisation of the dialogue games developed in the area of Informal Logic, which is an area of philosophy rich in models of communication and discourse, with a heavy focus on argument and “dialogue games”. If the dialectical model (e.g. dialogue game) is, as purported, a model of fair and reasonable dialogue, and if both the computer and students follow the dialogue, then fair and reasonable dialogue will ensue (Moore 1993). Our interest is to use a dialogue game as a vehicle for an educational human–computer dialogue system.

There are many types of dialogue interactions in which people reason together, such as debate, persuasion, inquiry and information-seeking (Walton and Krabbe 1995). The debating style of dialogue interaction is argued by Maudet and Moore (2001) to be important in critical thinking and developing debating and reasoning skills and is also suggested by Pilkington and Mallen's (1996) educational discourse analysis to be effective and to have a rich educational benefit. A particular concern with our research therefore is to investigate the issues surrounding a computer-based system for educational debate.

Earlier, we developed an amended dialogue model DE (Yuan 2004) based on DC (Mackenzie 1979) as the underlying model for our debating system. The motivation behind this development is that the underlying dialogue model of the debating system is required to have the ability to pick out fallacious argument and common errors when they occur during the course of debate. DE appears advantageous over DC in preventing the fallacy of question begging, inappropriate challenges and the straw man fallacy and appropriate handling of the issue of repetition (Yuan, Moore and Grierson 2003).

A particular concern with DE, however, especially from a computational perspective, is that it leaves much to the discretion of the user of the model. For example, after a challenge (why P?), various options are open: one can respond with a “no commitment” to P or a resolution demand (in some circumstances) or a support for P. Further, there is no guidance within the rules as to the content of the support. Similarly, after a withdrawal or a statement, there are no restrictions on the move types or move contents. All DE does is to legitimise a set of move types given the prevailing circumstances, and occasionally give some indication of the semantic possibilities. In a human–computer debate setting, it is therefore crucial that the computer is given some means of selecting between available possibilities, e.g. to maintain focus after a statement or a withdrawal, so that the produced moves are appropriate at the pragmatic level. This choice must be based on some suitable strategy.

Appropriate strategic knowledge is, then, essential if the computer is to produce high-quality dialogue contributions. The importance of strategies in dialectical systems has also been stressed elsewhere (e.g. Bench-Capon 1998; Walton 1998; Amgoud and Parsons 2001; Maudet and Moore 2001; Amgoud and Maudet 2002; Rahwan, McBurney and Sonenberg 2003). A set of strategies, enabling a computer to act as a debate participant was therefore proposed based on experimental study of the DC game with human participants (Moore 1993) and subsequently further developed in (Yuan 2004). However, the issue of whether the proposed strategy can in practice provide adequate services for a computer acting as a dialogue participant to produce good dialogue contributions cannot be settled on an a priori basis. The aim of this paper is to outline our work seeking to evaluate the proposed strategy.

To assess the appropriateness of a proposed strategy, Maudet and Moore (2001) suggest that the strategic heuristics need to be tested and that a convenient way to do this is via generation of dialogue by the computer itself. There are two possible ways to approach this: (1) allow two computational agents to engage in dialogue with each other and then study the results and (2) enable a human user to debate with a computerised debating system. Both approaches are seen as important to evaluate the proposed strategy from different perspectives. The former approach focuses on assessing whether there are unexpected new situations, requiring new heuristics, which have been missed in the current proposal and assessing whether the computationally generated dialogues are reasonably sound from a dialectical point of view. The latter approach, however, focuses on assessing the usability of the proposed strategy from the users’ point of view. An agent–agent assessment is necessary prior to user-based assessment to avoid issues such as missing heuristics and apparent flaws appearing in and interfering with more expensive user studies. Both approaches are therefore used in this study. A prerequisite for the study is the construction of suitable computational agents.

The remainder of the paper is organised as follows. First, we introduce the game DE and our current set of strategic heuristics. Secondly, we discuss the construction of a set of computational agents that can engage in debate with each other, generating dialogue transcripts for subsequent analysis. Thirdly, we discuss our human–computer debating system with which the user-based evaluations were carried out. We then discuss related work in this area and the significance of this work. We finally draw the conclusion and discuss our intended future work.

2.The dialogue model DE

The DE system is set up with two participants in dialogue with each other. Participants’ moves are regulated by a set of rules, which prohibits illegal events. The set of rules is outlined as follows (cf. Yuan et al. 2003).

3.Debating strategic heuristics

One of the primary motivations behind the development of our debating system, as argued in Section 1, is the expectation that it can be used to educational advantage – to develop students’ debating and reasoning skills and domain knowledge. In the context of an educational human–computer debate, the computer is ultimately intended to be not only a debate competitor but also an intelligent tutor. From an educational point of view, while intuitively one may wish the system to “speak the truth”, on the other hand, it could be argued that some sort of deception may be inherent in the definition of dialectical argumentation (Grasso, Cawsey and Jones 2000) and in the playing of devil's advocate, yet both of these may be educationally valuable (Retalis, Pain and Haggith 1996). A balance between trust and deception might therefore be required. It can be argued that the computer should be honest with respect to the publicly inspectable stores, since the system should be seen to be trustworthy. How, though, should the computer treat its knowledge base? The computer is required to have the ability to argue either as a proponent or as an opponent of the topic under discussion, and this implies that the computer's knowledge base can support both the opponent view and proponent view (see Appendix 1 for an example of the system knowledge base in the domain of capital punishment). As a result, the computer may constantly face inconsistent knowledge while making decisions (for example, it can find both support for and objection to the notion that capital punishment acts as a deterrent). In this situation, it is suggested that the computer is allowed to insist on its own view for the sake of argument even though it may have more reasons in favour of the user's view. Given the above discussions, the system is currently configured as what can be described as a partially honest agent. Against this profile of the debating system, a set of debating heuristics was proposed in (Yuan 2004) and is outlined below.

In the DE model, there are five dialogue situations that the computer might face, defined by the previous move type made by the user: a challenge, a question, a resolution demand, a statement or a withdrawal. Each therefore needs to be considered in relation to the strategic decisions the computer might need to make. It has been argued (e.g. Moore 1993) that these decisions are best captured at three levels.

Levels (1) and (2) refer to strategies which apply only when the computer is facing a statement or withdrawal, since in all other cases the computer must respond to the incoming move. Level (3) refers to tactics used to reach the aims fixed at levels (1) and (2) and applies in every game situation. These levels of decisions are discussed in turn below.

Level (1) decision concerns whether to retain the current focus or to change it. The decision, that is, involves whether to continue the attempt to substantiate or undermine a particular proposition. Moore (1993) argues that continuing to execute a plan of questions or addressing the previous move will guarantee that the current focus is retained but that it is possible not to directly address the user's latest utterance yet still retain focus. Moore further suggests that there is a presumption in favour of addressing the previous move, but that this presumption may be broken when the line of questioning is deemed a blind alley, or if a successful removal of the user's support has been made, or if, on regaining the initiative after a period without it, a resolution demand can legally be made.

The decision at level (2) considers whether to adopt a build or a demolish strategy. A build strategy involves seeking acceptance of propositions that support the computer's own thesis, while a demolish strategy seeks to remove the user's support for his thesis. The decision is needed only at the beginning of games and when level (1) decision involves a shift in focus. A demolish strategy could possibly be part of a broader build strategy, e.g. a goal-directed plan of questions building the computer's own view might involve removing some unwanted responses from the user. A building attempt might also be part of a broader demolish strategy, e.g. the computer is using a line of questions to build the case for P in order to attack the user's view ¬gP. Moore found no evidence to suggest a priority between the build and demolish strategy. In the current debating system, we give priority to a build strategy and make the computer try to open as many subtopics as possible, on the grounds that in an educational debate the aim is to expose the full complexity of the situation and this might be best achieved if the computer seeks to continue until all the knowledge base (henceforth referred as KB) has been explored. Moore also argues that the decisions at levels (1) and (2) heavily depend on the results of level (3) methods. In the current debating system, for example, the computer checks level (3) methods first; if there are level (3) methods available, level (1) and (2) decisions do not need to be applied; however, if there is no level (3) method available, level (1) and (2) decisions will come into play, in that level (1) decision may be to switch the current focus and level (2) decision is to build the computer's thesis if there are build methods available.

The third level of decisions applies to each of the dialogue situations. Level (3) heuristics for each dialogue situation are given in turn below.

4.Agent-based evaluation

This section discusses an agent-based evaluation of the strategy proposed in the previous section. We first outline the construction of the two computational agents that were used for generating dialogue transcripts. We then propose criteria against which the agent-generated dialogue transcripts are analysed. We end this section with the analysis of results.

4.1.Computational agents

Two computational agents (referred to henceforth as Simon and Chris) who conduct debate with each other, operationalising the dialogue model DE and the proposed strategy, have been built by the authors using the Java programming language and deployed on the Internet (http://staff.unak.is/not/yuan/games/simulation/dialogueSimulationSystem.htm). The game starts with one agent (say Chris) asking the other agent (say Simon) his opinion on the controversial issue of capital punishment and adopting the opposite position to engage Simon in debate on the issue. Chris can adopt either a proponent or an opponent role. That is, if Simon chooses to support the view of “capital punishment is acceptable”, Chris will adopt the opposite view of “capital punishment is not acceptable”, and vice versa. Both agents then engage in debate on the topic of capital punishment, given these initial positions on the issue.

The agent system contains five main units: the interface unit, the dialogue unit, the planning unit, the commitment unit and the knowledge base unit. The interface unit provides the system's user interface (Figure 1). It provides a dialogue history, which records the debate and commitment stores to show both players’ commitment store contents. In order to control the process of the debate, a New game menu item is available to start the debate, a Pause button is designed to temporarily stop a debate and a Continue button can be used to resume the dialogue. A Save as menu item is provided to save the dialogue history and both commitment stores as separate files for subsequent analysis.

Figure 1.

Computational agents user interface.

The dialogue unit can be regarded as the dispatch centre of the dialogue interactions. This unit consists of a dialogue manager, an output manager and a referee. The dialogue manager controls the turn taking of the interaction and is in charge of output manager, referee, the commitment unit and the planning unit. The output manager is responsible for updating the dialogue history. The referee is responsible for enforcing the DE rule set and for the termination of the debate. The original DE regime makes no stipulation re-winning and losing, but following Moore (1993), one agent will lose the debate when it has given up its thesis or explicitly committed to the opponent's thesis. It might be possible that one agent runs out of strategy but still adheres to its thesis; under this circumstance, the agent hands over its turn to its dialogue partner. If both agents run out of strategy and a winner is still not decided, the referee will call off the game (in effect, the dialogue ends in a stalemate).

The planning unit is responsible for generating moves in the light of (i) the KB, (ii) the prevailing state of both commitment stores and (iii) the dialogue rules. Each agent is equipped with a planner. A planner manages assertion, challenge, withdrawal, resolution and question strategists, each of which is designed to deal with a different dialogue situation following the set of heuristics discussed in Section 3. When the agent planner receives calls from the dialogue manager, it will check the current dialogue situation and schedule the corresponding strategist to produce a move. The agent's planner will then pass the move to the dialogue unit to make the agent's contribution.

In addition, there are five components (focus shift manager, build manager, demolish manager, plan generator and plan executor) that are designed to provide special services to the assertion and the withdrawal strategists. The focus shift manager will be called by the assertion or withdrawal strategist to decide whether to change the current focus. The build and demolish managers will be called by the focus shift manager to check whether there are methods available to either build its own positions or attack partner's positions. The plan generator is responsible for generating a set of propositions and forming a line of questions when required by the assertion or withdrawal strategist, the build manager or the demolish manager. The plan executor is responsible for executing a plan. The assertion and withdrawal strategists will constantly look up whether there is a plan under execution, if there is, then they call the plan executor to carry on its execution.

The commitment unit is responsible for updating both agents’ commitment stores. It contains a commitment manager and two commitment stores, one for each party. The commitment manager will update both agents’ commitment stores according to the DE commitment rules. Each commitment store is designed to have two lists of statements, those that have been stated and those that have been merely implicitly accepted. In order to distinguish them from each other, a statement that is only implicitly accepted is marked with an asterisk as shown in Figure 1.

The knowledge base unit consists of a KB manager and the KBs of the two agents. When the game starts, the dialogue manager will invoke the KB manager to initialise both agents’ KBs. The agents’ KBs contain a set of propositions and consequence relationships between these propositions. These relationships are based on a Toulmin-like structure (Toulmin 1958). The domain knowledge is formalised from Moore's (1993) experimental study of DC with human participants, and an example of such a knowledge base (the one used in the current study) can be seen in Appendix 1.

5.User-based evaluation

A prerequisite for a user-based evaluation is to construct a human–computer debating system. A fully functional human–computer debate prototype, operationalising the dialogue model DE and the proposed strategy (currently able to debate the issue of capital punishment) has been built and deployed on the Internet (http://staff.unak.is/not/yuan/games/debate/system.htm). An example user system interface can be seen in Figure 2. The user system interface provides a debate history to record the debate for subsequent analysis, commitment stores for both the user and the computer and input facilities for the user to make a move. Each commitment store is designed to have two lists of statements, those that have been explicitly stated by the owner of the store and those that have been merely implicitly accepted. In the current system, a statement that is only implicitly accepted is marked with an asterisk, as shown in Figure 2. The commitment stores are updated during the debate according to the DE commitment rules.

Figure 2.

Human–computer debating system user interface.

Turning to the user input facilities, a menu-based approach is adopted. Under this approach, the user needs to make a double selection, choosing from the available move types and then from the list of prescribed propositions from the domain under discussion. To prevent the user from constantly breaking the rules and to increase the learnability of the system, only the legally available move types are provided by the system before the student makes a move. Once the user has selected a move type, they need to select some propositional content. The system provides a number of means for doing this, depending on the nature of the move type. The details are as follows: (1) the move contents for resolution demand and challenge move types can be selected from the computer's commitment store; (2) the move contents for a withdrawal can be selected from the user's commitment store; (3) the move contents for assertion and question move types can be selected from the list of propositions (with the aid of the implies checkbox shown in Figure 2, the user may construct a conditional, e.g. PQ). The location of propositions on the screen is highlighted with a green-coloured border. In addition, the message bar at the bottom of the user interface provides dynamic instructions to support user input.

The system enables the user (S) to conduct a debate with it on the controversial issue of capital punishment. The computer (C) can adopt either a proponent or an opponent role. That is, if the user chooses to support the view of “capital punishment is acceptable”, the computer will adopt the opposite view of “capital punishment is not acceptable”, and vice versa. The system then engages the user in debate on the topic of capital punishment, given these initial positions on the issue. Further details of the system can be found from (Yuan, Moore and Grierson 2007a).

Two types of evaluations have been carried out: expert evaluation and user-based evaluation. The aim was to assess how acceptable, usable and potentially valuable this innovation was, prior to greater exploitation and subsequent further evaluations of its educational value. Three HCI experts were invited to evaluate the human-computer debating system. One expert preferred to evaluate the system cooperatively with the system author, in that the system author noted down the pertinent issues while the evaluator operated on the system (in effect, adopting a cooperative evaluation approach; Dix, Finlay, Abowd and Beale 2004). In addition, the expert agreed to take part in a short interview after the cooperative evaluation session. After the evaluation, the notes of this evaluation were formalised by the system author and emailed to the evaluator to check their accuracy. The two other HCI experts preferred to evaluate the system at their own convenience. The debating system was emailed to these experts. Formal feedback was emailed back to the system author after their evaluations.

Essentially, the expert evaluations give positive evidence concerning the usability of the system, in general, and of the DE dialogue model and the proposed strategy in particular. This is supported by the evaluators’ views on their experiences of operating on the system, such as “definitely easy for students who are familiar with computers”, “very straightforward to use it”, “no procedures annoyed me while operating on the system” and “the system's overall performance is acceptable”.

There are, however two weaknesses concerning the proposed strategy that were revealed. One participant reported that she found it rather uncomfortable when the computer constantly hands over its turn after a period of debate. She further suggested that “this is fine, to make me to explore more argument. I would say it depends on personality of the debate participants”. The second weakness is that the system fails to make a concession at the right time. The evaluator wrote: “after two long debates with the computer, it seemed to let me win. Though it is not clear why at that point it changed its mind. During these debates I thought I had the computer agree to a series of propositions that would lead it to change its initial position but it seemed to hold these incompatible ideas, without difficulty. When it did concede, it was a surprise to me”. This reflects the issue that there is no heuristic available for the computer to voluntarily concede a debate except when the user checks its thesis adherence. At some point, the computer should concede the debate voluntarily, i.e. when its thesis supports have all been removed from its commitment store and the user's thesis support added into its store, whereas currently the computer, without a heuristic for voluntary concession, simply hands over its turn to the user. Current work involves amending the system to cater for this concern.

A user-based evaluation of the debating system has also been carried out and documented in (Ævarsson 2006). Ten university students from the University of Akureyri, Iceland, who are interested in debate participated in the study. Three were from the computer science department, two from the education faculty and five from the business faculty. No participant took part in more than one study. There were two pilot studies, in order to determine the set-up of the experiment, followed by eight further studies. Each study was conducted in three stages: introduction, debate and interview. Prior to each study, the debating system was set up on the screen by the researcher. An English–Icelandic online dictionary was provided since the participants were native Icelandic and English was their second language. The introduction session involved the researcher briefing the participants about the purpose and procedures of the study. In the debate session, the participant was asked to conduct debate with the computer for 15 min in the pilot studies; this was extended to 20 min for the subsequent studies. After the debate session, a semi-structured interview was carried out concerning users’ experience of using the system. The dialogue transcripts were saved and the interviews were taped for subsequent analysis. The results are summarised in Table 5. The 10 participants successfully conducted debate with the system without difficulty. In total, 462 turns were generated. The longest debate took 79 turns and the shortest 29 turns. Incidents of users breaking the DE rules were very rare. Six participants did not break the rules at all. Two participants seemed to have changed their original view on capital punishment, eight of the debates ended with a stalemate and none of the participants made the computer concede.

The interview transcripts were analysed under four headings: system intelligence, user enjoyment, value of the system and the user interface issues. All of the participants agreed that the system is intelligent and a worthy debate component though some participants would like to see the system more aggressive and more attacking. Eight participants said they enjoyed the debates with the system, and they particularly liked the non-deterministic nature of the system's dialogue contributions. Two participants said they felt frustrated a little with the input facilities though they managed to debate with the system. All participants claimed they would like to debate with the system again were it available on the Internet. The value of the system was affirmed by the participants. All participants agreed that the system can be used to help them practise argumentation. One participant recommended that the system could be used as an aid to the Dialectics course. Participants from the education faculty said they would like to see the system tailored for child education in many areas.

The user-based evaluation revealed several concerns with the user interface. First, in the beginning, the participants had to spend some time in figuring out where the debate took place, whether it was in the student commitment store, computer's commitment store or in the debate history window. Once they had worked this out, most participants stayed focused on the debate history rather than the commitment stores. Participants expressed that they would prefer a one-window arrangement like the MSN communication programme rather than the current three-window set-up. Second, the participants found it confusing sometimes being guided to select move contents from the commitment stores on the top of the screen while the major input facilities were located at the bottom of the screen. Thirdly, participants were not happy with constantly clicking the Move Content Choice then scrolling down in order to find a suitable proposition. Finally, participants were sometimes confused with the statement with an * prefix in the commitment stores.

To address these user interface issues, the user interface of the system is being redesigned and tested further. The first three weaknesses are being addressed together by moving the commitment stores to the Move Content Choice panel which has been redesigned as a tabbed panel with three tabs: available propositions, user commitment store and the computer commitment store. Thus the user interface becomes a one-window set-up, and at the same time the space for the list of visible move content choice is increased. The final weakness is being addressed by colouring different types of commitments rather than simply using an * prefix.

6.Discussion

We have outlined our research aimed at the development and evaluation of suitable strategies for an interactive system offering dialogue involving competitive debate. Before discussing the significance of the evaluations reported above, there are three possible difficulties with the methodology that ought to be discussed. The first difficulty concerns the general approach of this study. Dynamic testing has been used (as opposed to a static analytical approach). For the dynamic approach, by its very nature, completeness is not possible (Dijkstra 1972). This nature is reflected in this study, for example, no resolution demand dialogue situations are generated for analysis, and the experimental set-up of the agent configurations does not consider the situation where both agents have completely different KBs. Dynamic testing, however, shows one step forward than static analysis by enabling us to assess the quality of the dialogue generated by the strategy. Any concern of the limitation of dynamic testing has been alleviated to some extent by (1) the combination of the static analysis of the strategies when they were developed in (Yuan 2004) and in (Moore 1993) and (2) the representative data generated for this study.

Second, it might be argued that only a small number of dialogue transcripts (three in total from three pairs of agents) have been generated for analysis. However, this study is intended not as a statistical enquiry, but rather as an investigation into the detail of the argument generated by the strategy. Further, 165 utterances are generated (DE4: 59; DE5: 52; DE6: 54). Each utterance needs to be considered in depth, and as a result this study does, it is held, provide sufficient data for the purpose of this assessment of debating strategies.

The third difficulty may be that there is a heavy reliance on judgements of quality by the author of the heuristics and the systems and that the criteria of quality are themselves intuitively formulated. The judgement issue may be endemic to the field, and similar criticisms could perhaps be levelled against much of the dialectics literature (Moore 1993). Further, computationally generating dialogues from dialectical theories may represent a step forward, and making the various criteria clear and explicit may well localise the issues to relatively narrow concerns at any one time, and this may detract from the judgemental element. In addition, these criteria have enabled us to provide a thorough analysis of the data collected, and to leave the results, and the data itself, available for independent inspection.

We argue, then, that the methodology adopted is sound. We believe that the work reported makes a valuable contribution to the fields of human–computer dialogue in general, and of strategies in dialectical systems in particular. Concerning the latter, we have proposed a set of strategies to be utilised with the dialectical system DE and provide a means to assess the appropriateness of the strategy. Further, since the agent systems and the human–computer system we have built can potentially be adapted to function with a different set of strategies, it potentially provides people working in the field of dialectical strategies with a test bed within which they can experiment with new strategies they develop (Maudet and Moore 2001; Amgoud and Maudet 2002).

Considerable research effort has been devoted to the formulation of strategies for dialectical systems. Grasso et al. (2000), for example, adopt, for their nutritional advice-giving system, schemas derived from Perelman and Olbrechts-Tyteca's (1969) New Rhetoric, and Ravenscroft and Pilkington (2000) have “a repertoire of legitimate tactics available for addressing common conceptual difficulties” (p. 283) that can be selected based on a strategic pedagogy developed through modelling effective tutor behaviour (Ravenscroft 2000). Amgoud and Maudet (2002) suggest meta-preferences, such as “choose the smallest argument”, to drive the choice of move, and Freeman and Farley (1996) delineate ordering heuristics as guidelines for selecting argument moves. Prakken (2001, 2005) suggests, in line with his dialogue framework, that the dialogue focus and dialectical relevance can be maintained by restricting dialogue participants to replying and backtracking to the previous move. Yuan, Svansson, Moore and Grierson (2007b) implement a probability-utility-based strategy, which enables a software agent to compute all the possible dialogue sequences and then select a legal move with the highest probability of winning an abstract argumentation game. Similarly, Oren, Norman and Preece (2006a) present a strategy for argumentation based on the principle of revealing as little information as possible, and Oren, Norman and Preece (2006b) discuss strategic possibilities when an agent's information varies in confidentiality. Noting that information might be so highly confidential that the cost of revealing it in the course of an argumentation dialogue would outweigh the benefit accrued from winning the dialogue, the authors present a heuristic to help an agent take account of such confidentiality-related costs in argumentation.

Our work adds to this body of research and is unique in that it concerns specifically strategies for computer-based debate. More generally, our work contributes to the field of human–computer dialogue. We have proposed a set of strategies for an educational human–computer debating system and our evaluations of the strategy in use reveal that the strategy is able to provide a good service for a computer to act as a dialogue participant. Our debate system is a unique system and therefore makes a contribution to the broadening of the human–computer interaction bandwidth, in general, and to the development of computer-based educational debate, in particular. Further, given the usefulness of a dialectical approach to interactive computer systems (Moore 1993; Moore and Hobbs 1996; Yuan 2004; Ravenscroft, Wegerif and Hartley 2007), any development of dialectical strategy per se potentially has a pay-off in terms of human–computer dialogue.

7.Conclusion and further work

We have proposed a set of strategic heuristics for a human computer debating system regulated by the dialogue game DE. An agent-based dialogue system and a human–computer debating system have been constructed to facilitate the evaluation of the proposed strategy. Both agent-based and user-based evaluations of the strategy in use have been carried out. The evaluations provide evidence that the proposed strategy can provide good service for a computer to act as a dialogue participant. The results are essentially favourable in demonstrating the innovation's acceptability and usability. The evaluations also provide evidence for the educational and entertainment value of the system. Several weaknesses of the dialogue strategy and user interface are revealed and discussed, and our immediate further work is to address these concerns.

There is also a variety of additional interesting ways in which the research can be carried forward. A limitation of the current study is that it is restricted to the consideration of a single strategy. The comparison is with moves generated in a random manner. This was necessitated by the absence of alternative strategies. When there is a refined version of the strategy available in the future, further comparative experimental studies of the strategy can be conducted, e.g. by comparing the current set and the refined set. A similar weakness is that our evaluative criteria need to be judgementally applied. Further work will involve refinement of the criteria, in line perhaps with Amgoud and Dupin de Saint-Cyr (2008).

Turning to our basic human–computer debating system, it can be enhanced to allow the user to question or challenge a conjunction of statements (e.g. P ^∧ Q) or conditional. Currently, the DE system is perhaps over-zealous at the challenge move, in that saying “why P?” removes commitment to P, whereas a dialogue participant may wish to remain committed to P but hear his interlocutor's reasons for P. Relaxing this rule could be implemented and tested. Further features, such as the system explaining its tactics and reasons for choosing one move over another to the user, along the lines of Pilkington and Grierson (1996), can also be implemented. The system could then be evaluated with a number of different domains of debate, e.g. abortion, politics, terrorism, to test the extent to which the design and knowledge representation are generic. This evaluation might be extended to encompass the use of the system to investigate pedagogic issues, such as the educational value of one to one debate, and how learners make inferences about the knowledge domain (Moore 1993). The evaluation could also be used to chart, through and across dialogues, how the way in which students engage in dialogue evolves. Ultimately, the system can perhaps be enhanced to keep track of student learning as such.

We are also planning to permit free user input for the debating system, initially via an option to enter fresh propositional content in addition to selecting from those made available by the system. This will enable us to build up the system knowledge base by adding new claims to it and, more importantly, to experiment with the extent to which the strategic heuristics can cope with such new input.

A further problem with our basic debating system, and indeed the underlying DE model, is that the move set, and in particular the question move, is restricted to bi-polar questions. Specifically, the absence of a move which enables debate participants to seek explanation of the substantive points being made, assumes that each participant fully understands all the points. In an educational context in particular, this is likely to be an undesirable restriction. Given this, our current research is seeking to extend the dialogue beyond debate per se and Naim et al. (2009) have addressed this.

The dialogue model and the strategy are currently represented informally using structured English and are hard-wired into the program logic by the developer of the system. Ideally, the dialogue model, strategy or even the knowledge base of the system could be written and modified by the users of the system and then directly translated to the program code. A formal representation is required for this. This is a general software engineering issue in the field of computational dialectics. A plausible investigation might be along the lines of (Bench-Capon, Geldard and Leng 2000; Reed 2006; Yuan, Moore, Reed, Ravenscroft and Maudet 2011) where each move act is specified using a pre- and post-condition pair.