A two-level approach to generate synthetic argumentation reports

1.1.Argument mining challenges

Argument mining is an emerging research area which introduces new challenges in natural language processing and generation. Argument mining research applies to written texts, e.g. [10,13], for example for opinion analysis, e.g. [22], mediation analysis [9], scientific text analysis [8], advertising [2], or transcribed argumentative dialog analysis, e.g. [3,21]. The analysis of the NLP techniques relevant for argument mining from annotated structures is reviewed and analyzed in e.g. [14]. Goal-based reasoning for argumentation analysis is developed in [24]. Annotated corpora of various types are now available, e.g. the AIFDB dialog corpora or works reported in [23], among many others. These corpora are very useful to understand how argumentation is realized in texts and dialogs, e.g. to identify Argumentative Discourse Units (ADUs), linguistic cues at stake [12], and argumentation strategies, in a concrete way, possibly in association with abstract argumentation schemes, as shown in e.g. [6]. Finally, reasoning aspects related to argumentation analysis are developed in e.g. [7,17] and [25] from a formal semantics perspective with e.g. a study of concessive and contrastive connectors.

In opinion analysis, the benefits of an argument mining synthesis are not only to identify the customers satisfaction level, but also to characterize why customers are happy or unhappy. Abstracting over arguments allows one to construct summaries and to infer customer preferences or value systems (e.g. low rates are preferred to localization or quality of welcome for some categories of hotel customers).

In [20], a corpus analysis identifies the types of knowledge and inferences that are required to develop argument mining within a symbolic perspective. The structure of this corpus is briefly reported in this paper. Then, we show, on the basis of that corpus, that the Generative Lexicon (GL) [16] could be an appropriate linguistic and conceptual model and organization, sufficiently expressive, to characterize the types of knowledge and lexical data that are required to accurately identify propositions related to an issue. This model develops the crucial conceptual aspects that relate propositions to an issue. The conceptual dimensions attacked or supported in arguments are indeed related to the roles, functions, and purposes of the head entities in the issue at stake, or how these entities came into being, were created or produced. The GL Qualia structure is therefore considered here for the conceptual and linguistic structure it offers, independently of the initial uses developed by its initiator, in particular to model language variations and metonymy construction [16].

1.2.Natural language synthesis vs. summarization

In natural language generation, the main projects on argumentation generation were developed as early as 2000 [26] and [7]. Focus was on the production and the organization of arguments from a semantic representation, via some forms of macro-planning, an essential component of natural language generation of texts. A synthesis, and in particular an argument synthesis is a much simpler operation which consists in grouping already existing propositions related to an issue on the basis of their similarity in order to enhance readability. It is therefore more superficial that the generation of arguments.

While there are currently several research efforts to develop argument mining, very little has been done recently to produce a synthesis of the mined arguments that is readable, synthetic enough and relevant for various types of users. This includes identifying the main features for or against a controversial issue, but also tasks such as eliminating redundancies and various types of fallacies which are less relevant in a synthesis that is aimed at being as neutral as possible. This synthesis may have the form of a short text, possibly including navigation links, or it may be represented by means of a network or by various types of graphical representations that organize the reasons or justifications for or against an issue. The major problems are readability, granularity and conceptual adequacy with respect to users.

In [19], we show how propositions that have been mined can be organized in hierarchically structured clusters, according to the conceptual organization proposed by the Generative Lexicon, so that readers can navigate over and within sets of related arguments. This approach turned out not to be synthetic enough, since over 100 reasons can be mined for a given issue, making the perception of the main attacks and supports quite difficult. Nevertheless, this initial approach allows the construction of an argument database with respect to an issue which is useful to readers who wish to access the exact form of arguments that have been mined. This is illustrated in Fig. 1, where the relation of the concept to the issue (part of, purpose, etc.) is specified to enhance the structure of the description. This figure simply shows a form of synthesis readers would like to have, independently of any knowledge representation consideration.

Fig. 1.

Illustration of a synthesis.

The present contribution goes one step further: it aims at producing a synthesis that is short and efficient where the concepts present in the GL Qualia structures are used to abstract over sets of propositions that support or attack an issue at stake. The structure of clusters of propositions is kept and made accessible via links from this synthesis.

This contribution to natural language argumentation synthesis is not really a summarization task, as e.g. developed in [11]. In our approach, no text or document is reduced to produce a summary. The synthesis that is proposed is simply a two level re-organization task of the reasons mined for or against an issue, that involves forms of clustering.

From that perspective, this work could be viewed as a preliminary step to a summarization procedure. A real summarization task would involve e.g. identifying prototypical arguments and themes, and then constructing summaries for each group of related arguments, but this is beyond the present research.

In terms of feature classification and relevance, we show below that the concepts used in the Qualia structure of the Generative Lexicon are appropriate for this synthesis task. The concepts in the Qualia are defined from a domain ontology, similarly to the features evaluated in most opinion analysis systems. We show that these features can be used as entry points to the cluster system and as a measure of the relevance of a proposition with respect to the issue at stake. A challenging point is that these concepts and their internal organization must correspond as much as possible to the user perception of the domain to which the issue belongs, otherwise the synthesis may not be so useful.

2.1.Corpus analysis: Identification of the required knowledge

To explore and characterize the forms of knowledge that are required to develop argument mining in texts, we assembled and annotated four corpora based on four independent controversial issues as reported in [18]. The main results are summarized below. These corpora are relatively small. They are designed to explore the problem, and to elaborate a model, not to design a comprehensive argument mining system. The texts considered are extracts from various sources, e.g.: newspaper articles and blogs from associations. Issues are:

The total corpus includes 51 texts, a total of 24500 words for 122 different propositions that support or attack one of the above issues. From our manual analysis, the following polarities are observed: attacks: 51 occurrences, supports: 32, argumentative concessions: 17, argumentative contrasts: 18 and undetermined: 4.

Our analysis shows that for 95 propositions (78%), some form of knowledge is involved to establish an argumentative relation with an issue. An important result is that the number of concepts involved is not very large: 121 concepts for 95 propositions over 4 domains. Even if the notion of concept remains somewhat vague, these results are nevertheless interesting to develop large argument mining systems. These concepts are mainly related to purposes, functions, parts, properties, creation and development of the concepts in the issues. These are relatively well defined and implemented in the Qualia structure of the Generative Lexicon, which is the framework adopted in our modeling.

2.2.An introduction to the generative lexicon

The Generative Lexicon (GL) [16] is an attempt to structure lexical semantics knowledge in conjunction with domain knowledge. Its main structure is the Qualia structure. The notions used in the GL Qualia structure are not new: the roles postulated in the Qualia structure are derived from the Aristotelian class of causes: constitutive for material, formal for formal, telic for final and agentive for efficient. These roles do not overlap with the notions of grammatical roles or semantic roles. They are much more powerful from a semantic and knowledge representation point of view.

In the GL, the Qualia structure of an entity is both a lexical and knowledge repository composed of four fields called qualia roles:

The vaccine against Ebola is necessary.

The main concepts in the Qualia structure of the head term of (1), vaccine are organized as follows:

The main concepts to be found in the Qualia structure of Ebola are:

The terms, predicates or constants, found in the different roles of any Qualia are defined on the basis of a domain ontology, when it exists, or via bootstrapping techniques on the web, if such an ontology does not exist for this domain. Besides these predicates and constants, we have included formula as attribute values. In the above example of Ebola, event variables Ei have been added to deal with complex events. In the Qualia of vaccine, subtypes (e.g. MAIN, MEANS) have been added to provide a more accurate structure to the main four roles. More examples are available in [18].

Qualia structures can be hierarchically organized, as in any ontology. Vaccine is a kind of medicine, it therefore inherits of the properties of a medicine, i.e. the predicates present in medicine are inherited, unless some blocking is formulated. Similarly, Ebola is a kind of disease, therefore it inherits of the properties of a disease. This rich organization greatly simplifies the description of Qualias. Some Qualia structure resources are available as payware at ELRA, from the SIMPLE-PAROLE EEC project.

Finally, from the two above Qualias and via formula expansion, the formal representation of the controversial issue is (using the modal symbols of necessity and possibility):

2.3.Using knowledge for argument mining

Originally, the Qualia structure was designed to characterize sense variations around a prototypical (or domain-stabilized) one, and the large number of potential combinations of NP arguments with predicates, in particular verbs. This was implemented via a mechanism called type coercion [16]. In our view, this is a specific interpretation of a more global typology of object descriptions.

In our approach, we view the Qualia structure as a means to represent knowledge associated with concepts in a functional way, via telicity (an subtypes of telicity), various types of functional and structural parts, and the way an object was created. This view allows us to introduce more flexibility and higher-order representations in Qualias, in particular complex structures such as formula, modalities, etc. instead of just atomic concepts. Manipulating such structures is more complex, but corresponds to the needs in terms of representation. Reasoning with these complex forms is addressed in [19].

Other types of resources such as FrameNet, WordNet or VerbNet do not contain the information found in Qualias, which is essential for argument mining. These latter resources are structured around predicative forms and mainly describe the type of arguments and adjuncts predicates can take and how they are combined. VerbNet introduces semantic representations based on primitives which may be of interest for our approach as a way to normalize the complex representations we have implemented and, possibly, the atomic concepts themselves.

In terms of data completeness, it is clear that Qualia descriptions will never be comprehensive knowledge repositories for a given concept, with all its facets. In our approach, due to a lack of existing resources, Qualias are mostly described manually. Even via the use of bootstrapping techniques, it is clear that the Qualia of a concept C (e.g. vaccine) essentially contains the most typical features. These are encoded via concepts, which themselves can originate Qualias. An incremental automatic acquisition of Qualia features would be crucial and helpful. This raises complex problems such as consistency or granularity management which are outside the scope of this article. Some elements for automatically acquiring Qualias are developed in [4]. In this article, focus is on the model provided by the GL Qualia, which seems to be particularly relevant for knowledge-based argument mining, without excluding the use of other paradigms to specify e.g. the contents of constitutive roles (from WordNet) or verb arguments as implemented in VerbNet via primitives.

3.1.Network construction

A Qualia Qi describes a number of functional and structural features of a concept. It can be formally defined as follows:

A network of Qualias, for a given controversial issue, is then defined as follows:

This network of Qualias forms the backbone of the argument mining system. This network develops the argumentative generative expansion of the controversial issue. This network is also the organization principle, expressed in terms of relatedness, that guides the generation of a synthesis where the different facets of the Qualias it contains are the structuring principles [19]. Natural language words or expressions that lexicalize each concept are associated with each network nodes.

An important concern is to evaluate if and how this network defines a kind of ‘transitive closure’ that would characterize the typical and most frequent concepts that appear in arguments that support or attack an issue. Obviously, unexpected arguments may arise with concepts not in this network, probably with a lower frequency and recurrence. The propositions which are not found are rather unexpected, but of much interest. These may indeed be related to specific local social or historical contexts. For example, propositions such as: vaccination prevents bio-terrorism, vaccination raises ethical and racial problems are found. These propositions are stored in a specific cluster called ‘Other’, so that they can be accessed from the synthesis.

The total number of concepts at stake in arguments for an average size issue, such as issues (1) and (3), is about 40 to 50 concepts, depending on the granularity implemented in Qualia. Quite different occurrence frequencies are observed. A rough estimate indicates that about 85% of the propositions related to an issue can be recognized on the basis of these concepts. The ‘transitive closure’ induced by this network is obviously not complete, but quite relevant.

3.2.An illustration

Let us now illustrate the construction of this network. For example, from ‘vaccine’, two nodes of the constitutive role are candidates:

The concepts in the formal and telic roles are stored in T1:

Similarly, test(T, X), in the agentive role of vaccine(X), applied to vaccines (and medicines more generally), originates a node in T, and additional nodes in T1 from its non-terminal concepts:

From these two examples, T (root concepts) and T1 (immediate daughter concepts) include the following elements, assuming that variables are bound within each set:

These concepts are associated with lexical terms or expressions, such as:

Besides other elements, such as evaluative expressions, these lexicalizations are used to detect the utterance relatedness to the controversial issue and to possibly qualify it as an argument. The linguistic structure of propositions that may attack or support an issue is described in [20].

Then, propositions may attack or support concepts present in T or T1, such as the evaluation of the protection offered by the vaccine or the test protocol that has been used.

4.1.Main arguments found in texts

Let us consider the arguments found in texts for issue (1) that must be included in a synthesis. Arguments mainly attack or support salient features of the main concepts of the issue or closely related ones by means of various forms of evaluative expressions. Contrasts in our terminology include propositions which are weak attacks, whereas concessions are weak supports. Among 50 non-overlapping arguments, the main arguments associated with issue (1) are, omitting associated discourse structures:

4.2.Structure of a argumentation synthesis

Let us now characterize the form of the synthesis, as it can be derived from the examples given in Section 4.1. A synthesis of the examples given in Section 4.1 can be organized as follows, starting by the concepts which appear in the issue (root concepts in T) and then considering those, more remote, which appear in the derived concepts constructed by the network of concepts, e.g. those in T1.

To make the synthesis readable and informative, between parentheses, the total number of occurrences of arguments mined in texts for that concept is given as an indication. The goal is to outline strengths, recurrences and tendencies. This however remains informal because occurrence frequencies are very much dependent on how many texts have been processed and how many arguments have been mined. This number is also a link that points to the arguments that have been mined in their original textual form. For each line, the positive facet is presented first, followed by the negative one when they exist, independently of the occurrence frequency, in order to preserve a certain homogeneity in the reading:

Each line of the synthesis is produced via a predefined language pattern. This is developed in the next sections. The comprehensive list of arguments is stored in clusters (Section 4.4) and are accessible via navigation links, represented in this synthesis by means of underlined numbers.

An alternative synthesis approach would have been to select for each concept an argument that is prototypical paired with a link to the other arguments related to the same concept. This approach sounds less artificial than the generation of abstract sentences, but (1) it is difficult to characterize what a prototypical argument is and (2) the formulation we propose is a priori shorter, more neutral and standardized. However, we feel that exploring other forms of synthesis, in close cooperation with users, should be investigated. For example, forms of rhetoric could be incorporated for non-neutral synthesis. Introductions to these aspects can be found, among many others, in [15], in [1] specific cases can be found in a number of articles of this journal.

4.3.Input data: Annotated arguments

The output of the mining system, which is the starting point of the synthesis construction, includes the following attributes [18], associated with each argument:

The ‘ConceptsInvolved’ attribute is structured from the root node, as a kind of path, so that the concept that is involved is clearly identified. This attribute may contain an ordered list of paths if several concepts are involved. A typical path is a sequence:

where the Concept is a predicate or a constant of a Qualia structure found under the role ‘Role’. For example, the concept of ‘protocol’ is defined as follows:

since ‘protocol’ is a concept associated with the constitutive role of the concept ‘test’ in T1. Besides a non-ambiguous identification of the concept ‘protocol’, this path can be used as (1) a way to structure a synthesis and (2) a way to provide some explanation of why an utterance is an argument by outlining its relation(s) with the root concept.

In terms of annotations, for example, Proposition 11 attacks the claim on vaccination: Even if the vaccine seems 100% efficient and without any side effects on the tested population, it is necessary to wait for more conclusive tests before making large vaccination campaigns. The national authority of Guinea has approved the continuation of the tests on targeted populations.

is composed of an kernel (it is necessary to wait for more conclusive tests before making large vaccination campaigns) modified by two discourse structures. This proposition is tagged as follows:

Discourse structures are those defined within the RST framework (http://www.sfu.ca/rst/). At this stage no meta-data is considered such as the date of the statement or the author status. This notation was defined independently of any ongoing task such as ConLL15.

4.4.Clustering arguments

The last step before the production of a synthesis is to cluster arguments according to their relatedness with respect to the controversial issue. The network of Qualias together with the attributes ‘relationWith’, ‘Polarity’ and ‘ConceptsInvolved’ are the most crucial elements in the annotation system to generate clusters that correspond to the domain conceptual organization. Each concept of the Qualia network potentially defines a cluster.

The generation of clusters proceeds as follows:

Let us consider the example on vaccination, then a set of clusters would be composed of the following arguments:

5.1.The lexico-grammatical generation system

The synthesis of arguments is based on abstract linguistic patterns defined as follows:

The symbol HeadConcept is the lexicalization of the rightmost (or leaf) concept in the attribute ‘conceptsInvolved’. For example, in:

the rightmost concept is ‘Test(T, Z)’, its prototypical lexicalization, stored in a lexicon, is ‘test’. For example:

The function lexicalization(Word, Predicate) extracts the lexical item from the appropriate lexical entry. In this lexical entry, the last argument contains closely related lexical terms. These are used in particular to detect relatedness.

When complex forms are given as attributes, such as ‘avoid(T, dissemination(X))’, then the content that is lexicalized corresponds to the whole formula: ‘avoids the dissemination’. In our current approach such formula are generated on the basis of predefined patterns, for example:

In our example, avoids dissemination would be generated. Some morphological features need to be taken into account in the generation rules.

Finally, when the path given in ‘conceptsInvolved’ is long (from 2 concepts), in order to avoid any ambiguity, a lexicalization of the whole path is produced, e.g. for the example above: test of the vaccine instead of test alone, using the basic compound NP pattern:

Next, the symbol Be/Predicate in patterns (1) and (2) above entails the lexicalization of the main predicate of the sentence. It is either the neutral ‘be’ (is, are) or a specific lexicalization if the attribute value that is considered includes a higher-level predicate such as prevent, evaluate, allow, avoid. In that case, Be/Predicate is the lexicalization of the higher-level predicate found in the attribute. For example, the attribute: ‘evaluate(T, protection(X,Y,A))’ is lexicalized by: evaluate the protection of. The variables X, Y and A are lexicalized by other parameters in the patterns (1) or (2).

When there are supports and attacks, this predicative form appears as such and is then modified by a negation so that supports and attacks can be differentiated in the synthesis, using the following patterns:

Supports:

Attacks:

In these patterns, Predicate represents the whole attribute. For example, an attack of the function of a vaccine avoid(X, dissemination(D)) produces the expression:

does not avoid the dissemination.

Next, the symbol (Stats) indicates the number of arguments that have been mined with the ‘conceptInvolved’ path considered, i.e. the number of arguments in the corresponding cluster (Section 4.4). These statistics are indicative since they depend on the volume of text that has been mined. They also do not account for the strength of the argument. (Stats) is a link to the corresponding cluster of arguments that the reader may wish to inspect.

Next, the symbol Evaluative in the patterns (1) and (2) is an evaluative expression, often a scalar adjective, possibly modified by a negation depending on its polarity, the existence of an antonym (which could be prefered to avoid negation) or the polarity of the argument. Adjectives, as well as nouns, have these semantic characteristics stored in their lexical entry. The lexicalization of Evaluative is defined as follows:

Finally, the symbol AttributeLexicalization is the lexical item that corresponds to the concept. In our approach this lexical item is stored in a lexicon where lexical entries include lexical items and their associated logical representations, as described above. When the attribute is propositional, the same strategy is used, in that case, an expression is produced via a pattern instead of a single item. This is the case for example with ‘develop(T,X)’ which gets the realization to develop.

This synthesis generation system is quite simple at the moment. Besides domain lexical entries, related to the concepts in the Qualias (between 50 and 100 lexical entries depending on the issue), the system currently uses 22 patterns and sub-patterns that allow to produce the constructions presented in Section 4.2. These patterns are stable for the type of issues we consider, which are simple, with arguments which are direct and concern a single concept of the network.

This generation system needs further investigation for more abstract issues or complex situations such as controversial dialogs. In this first stage, only the relations between the controversial issue and arguments have been investigated. In a ‘real’ argumentation, arguments may attack or support other arguments instead of the issue. It is not clear at the moment whether the same generation procedure can be used. Probably, to keep the synthesis readable, additional devices such as navigation links would be needed to indicate that an argument gets supports or attacks from others.

The system described here is relatively simple to implement. A first implementation has been realized with the logic-based platform <TextCoop> we developed for discourse analysis. This platform can also be used for language generation since it is fully declarative and partly reversible. However, the strategy used in <TextCoop> needs to be revised so that the simplest structure is generated first. The parsing strategy that is used in <TextCoop> is indeed a priori oriented towards language parsing.

5.2.Features of an evaluation

This approach to argument synthesis generation is relatively simple and straightforward. The two levels: synthesis and links to the exact arguments stored as clusters seems to be a good compromise between proliferation of data and over generalization via a few synthetic lines. Elaborating an appropriate level of synthesis and the way to realize it linguistically needs a lot of experimentations and tunings.

An evaluation of the current system, in order to identify improvement, must be realized along at least the following main features:

A higher level evaluation concerns the adequacy of this synthesis for professionals who want to access to an argument synthesis, where arguments do correspond to their own view and to the analysis they have of the domain. For example, in the hotel and restaurant domains, the features at stake are predefined by professionals in consumer evaluations. For more abstract or less common areas such as the issues given in Section 2.1, there is a need to make sure that the concepts developed in the Qualias do correspond to the vision of professional users, otherwise such a system will not be of much practical relevance.

We also feel there is no unique form of synthesis: several forms of synthesis could be foreseen that would depend on the reader’s interests and profile. A real evaluation of the system presented here requires the development of adequate protocols to measure the relevance of various forms of synthesis, e.g. more abstract, more or less concise, using various types of concepts, with appropriate lexicalizations. This can only be done with large and diverse populations of users over several domains.