Argument schemes and visualization software for critical thinking about international politics

1.Introduction

Critical thinking about international politics often involves reasoning about the beliefs, goals, appraisals, actions, and plans of actors such as countries, governments, politicians, etc. We analyzed arguments in interpretive reports about international politics, in order to develop a prototype argument diagramming tool for this domain, AVIZE (Argument Visualization and Evaluation). The tool is designed for creating arguments based upon evidence of varying plausibility collected from sources of varying reliability. In the era of “fake news”, such a tool could be of use in on-line environments for citizen engagement [4] and in educational settings [12,13]. In addition, it could be useful to an analyst of international affairs as new information is learned that changes the acceptability of a previously created argument. AVIZE provides a set of presumptive argument schemes [19] as cognitive building blocks for constructing argument diagrams. While some currently available tools provide argument(ation) schemes, e.g., subsets of the ones listed in [19], they are not tailored to the domain of international politics. Finally, AVIZE helps the user to evaluate his arguments by enabling him to visualize arguments and counterarguments for a position. This paper describes the argument schemes11 we defined for this domain and the design of AVIZE.

2.Argument schemes

This section describes the argument schemes included with the AVIZE tool,22 which are based primarily upon our manual analysis of arguments in an article on the Russian government’s strategy for increasing Russia’s global influence [20]. Analysis of arguments in the 33-paragraph article revealed fifteen instances of eight schemes, illustrated in Table 1. Analysis of several other articles, including one on alleged Russian interference in the 2016 U.S. presidential election [1], revealed instances of three additional schemes that are included in AVIZE’s scheme set. Note that Table 1 does not show the discourse structure of text segments in that article as might be represented in a theory of discourse coherence such as Rhetorical Structure Theory (RST) [10]. To reconstruct the arguments it was necessary to go deeper than the surface text level; it was necessary to consider the intended meaning of the text, to disregard presentation order, to identify implicit premises and conclusions, and to ignore or abstract over some text segments. The goal of our analysis was not to develop a document annotation system for use in machine learning experiments on argument mining as in, e.g., [11,16]. (The design of annotation systems for machine learning is much more highly constrained than necessary for our goals.) The current schemes may be refined after experience in use of AVIZE, and it is likely that additional schemes will be added over time. (For more information on the intended use of AVIZE, see the next section.)

The argument schemes are described in Tables 2 and 3. Examples of certain schemes (i.e., those that were not illustrated in Table 1) are from [1]. The schemes in Table 2 involve making sense of an Actor’s observed actions, while those in Table 3 are for planning the Protagonist’s actions. Since many of the schemes refer to adversarial situations, Protagonist and Actor are used to distinguish participants, e.g., the U.S. and Russia, respectively. For the sake of clarity to AVIZE’s users, the descriptions of schemes are phrased as simply as possible, i.e., avoiding awkward (but more precise) locutions such as ‘did/does/will do’ and ‘Act or Acts’. For similar reasons, Inferred Positive/Negative Appraisal is described as two schemes, despite their analogous descriptions. Another problem when providing descriptions of schemes is deciding how to divide information among the premises. For example, the three premises (1) Actor has Goal, (2) Actor does Acts and (3) The Acts are consistent with Goal, could be combined into one or two premises in the description of the Plan Inference scheme. If the granularity were too fine, users might find it laborious to construct an argument; if it were too coarse, a user who wished to dispute part of a premise would have no way of explicitly showing which part is in dispute.

AVIZE provides a list of critical questions, shown in Table 4, for each scheme listed in Tables 2 and 3. The critical questions can be posed to challenge arguments created from a scheme. In addition to the questions in Table 4 every scheme includes two critical questions: How reliable is the source of each premise? How likely is each premise? Unlike the schemes, the critical questions were derived primarily by introspection and consideration of critical questions listed in [19] and [3]. Note that most of the critical questions for the schemes described in Table 2 require the Protagonist to infer the Actor’s state of mind, i.e., goals, beliefs about the effects of acts, motivations, preferences, and assessment of benefit-to-cost. (Here the term ‘benefit-to-cost’ is used to suggest any kind of assessment of the consequences of actions, including the degree to which the action promotes or demotes some value [2].)

3.AVIZE design

The purpose of AVIZE is to aid users (e.g. college students or analysts preparing interpretive reports) in the construction and self-evaluation of real-world arguments in the domain of international politics. Unlike tools developed for text processing research, AVIZE is not designed for use as a corpus annotation tool. Unlike many educational argument modeling systems, AVIZE is not designed for helping users to analyze and visualize arguments in existing documents. AVIZE is designed to support critical thinking by a process in which users must evaluate evidence from a variety of sources such as print or broadcast news organizations; construct arguments in their own words; and consider potential challenges to their argument and defend their argument against those challenges.

The interaction design goals are to allow users to employ AVIZE with little or no training, to efficiently assemble argument diagrams using argument schemes for this domain, and to easily spot strengths and weaknesses of their arguments through visual means. The tasks that AVIZE is designed to support are shown in detail in Fig. 1. Note that the tasks need not be performed in the order shown and some are optional.

Fig. 1.

Task analysis for AVIZE.

AVIZE was designed for a desktop computer-sized display area and to provide a drag-and-drop style of interaction. As shown in Fig. 2, data items and (collapsible) metadata are presented in a (collapsible) panel on the left side of the screen.33 Collapsible argument scheme descriptions44 (each which, when opened, includes premises, conclusion, critical questions, and an example) are presented in a (collapsible) panel on the right side. The center area is used for constructing arguments. In Fig. 2, by dragging data items from the left panel to the center panel, the user has created a clump of evidence which contains conflicting data items on the theme of a possible Martian invasion of Earth. According to one source, hikers found a Martian space ship in the Nevada desert; however, according to two other sources the alleged spaceship debris can be explained away. Nevertheless, the user decides to create a proposition (by typing into an empty “proposition box” in the argument workspace) that Martians have landed on Earth and attaches it to the evidence clump. In this way, all evidence considered (both supporting and opposing) is available for examination.

Fig. 2.

AVIZE screen after user selects several pieces of data as evidence for/against a possible Martian landing on Earth.

Suppose that next the user hypothesizes that the Martian landing is part of a Martian plan to take over Earth, and so he creates a new “proposition box” in the argument workspace stating that Martians have invaded Earth as part of a plan to take over Earth. In order to build an argument for this thesis, he drags the Inferred Plan argument scheme from the right panel (as shown in Fig. 2) onto the argument workspace, causing AVIZE to automatically create a box-and-arrow-style template to contain an instance of that scheme. Then the user drags his thesis proposition onto the conclusion of the template, and the evidence-based proposition that Martians have landed on Earth onto the Actor does Acts premise of the template. Now he must fill the remaining two premises, the Actor has Goal premise and the Acts are consistent with that Goal premise. He enters the proposition that Martians want to exploit Earth into the Goal premise. Then he attaches two conflicting pieces of evidence from the left panel to it: (1) Martian leaders told their fleet that they want to take Earth’s coal and iron and use Earthling’s to mine it (which, according to the metadata, was from an intercepted communication), and (2) We come in peace (which, according to the metadata, was broadcast to Earth by the Martians). Although the evidence is conflicting, he has decided to credit the first data item over the second one. Lastly, he fills the consistency premise of the template in a similar manner after seeing the data item that Martians took over Earthlike planet Threa in another galaxy in order to enslave its population and exploit its resources. The final argument is shown in Fig. 3.

Fig. 3.

AVIZE screen showing argument about Martian plan to invade Earth.

Using drag-and-drop operations, a user can combine arguments to create a network of supporting and opposing arguments, as shown in Fig. 4. Arguments may be “chained”, i.e., the conclusion of one argument can serve as the premise of another argument. Multiple arguments for the same conclusion can be linked together. Critical questions can be invoked from a menu by clicking on an argument scheme name in the diagram. Premises and conclusions may be challenged by counter-arguments. Note that counter-arguments and critical questions are linked to other arguments via different icons (a shield and question mark, respectively). In this diagram, the evidence (and its metadata) has been collapsed. However the number of pieces of evidence for the lowest level premises is still shown. Lastly, a user can attach a numeric assessment of likelihood and/or a comment to any node, and can toggle the display of these. In addition to the numeric display, likelihood is automatically depicted by color on a scale from red (0.0) to blue (1.0). In Fig. 4, the user has assessed that the premises of his argument are highly to moderately likely (0.9, 0.5, 0.8) and that a possible counterargument has a low probability (0.3). Also, his argument has been weakened by the answer to a critical question.

Fig. 4.

AVIZE screen illustrating network of supporting and opposing arguments.

4.Related work

Many computerized argument diagramming or mapping tools have been developed, e.g., Compendium [19], Rationale [14,17], Online Visualization of Argument (OVA) [15], and LASAD [9]. Unlike these, AVIZE was not designed for use as a corpus annotation tool or to help users to analyze and visualize arguments in existing documents. AVIZE is designed for helping users to construct and challenge their own arguments based upon evidence of varying plausibility from sources of varying reliability. Another key difference is that AVIZE provides presumptive argument schemes (with critical questions) tailored to the domain of international politics.

Most of the argument schemes of interest in this paper are closely related to the field of automated recognition of an agent’s plan in artificial intelligence and natural language processing [5]. The earliest work in that field used heuristic rules describing the relationships among an actor’s beliefs, goals and actions to infer his plans. Due to its computational complexity that approach to plan recognition has been supplanted with probabilistic approaches. However, the heuristic rules for plan recognition resemble aspects of the argument schemes we have identified in Table 2. Furthermore, a Behavior Pattern argument could be described as inferring plans created by cased-based-reasoning, i.e., by adapting old plans [8]. Some modern AI planning systems incorporate appraisal theory into planning, e.g. [6]. Incorporating that perspective, an Increasing Boldness argument involves not only reasoning about an actor’s plan, but also the actor’s beliefs about the protagonist’s response to the plan; and Inferred Positive/Negative Appraisal involve reasoning about an actor’s appraisal of the effects of certain acts.

Table 3 presented schemes for arguing about what the Protagonist should do, i.e., sharing the perspective of automated planning in AI. Two of the schemes, Practical Reasoning and Avoid Negative Consequences (of Not Doing), are similar to Practical Reasoning and Argument from Negative Consequences, respectively, e.g. as described in [19]. Our Coercion scheme (in Table 2) is related to Argument from Threat [18]; its counterpart, Resist Coercion (Table 3), is a more specific version of Avoid Negative Consequences. Bex et al. [3] present a unified scheme for practical reasoning, which can be used either for practical reasoning (planning) or abductive reasoning (plan inference).

5.Conclusion and future work

We have described the argument schemes and design of AVIZE, a tool for constructing arguments, represented as diagrams, in the domain of international politics. The set of argument schemes was identified by analysis of arguments in representative documents and descriptions of the schemes are tailored to this domain. A prototype version of AVIZE has been implemented as a Java application and has been released as open-source software.55 Note that any sets of data and argument schemes, in appropriately XML-formatted files, can be accessed by AVIZE. Thus, the tool itself is not limited to use in this domain. No formal evaluation of AVIZE has been performed yet. In future work, we would like evaluate the effectiveness of AVIZE in educational settings. Also, future versions may add support for collaboration and debate among multiple users.