Knowledge-driven argument mining based on the qualia structure

3.1.Corpus characteristics and construction

To explore and characterize the forms of knowledge that are required to develop argument mining in texts, we constructed and annotated four corpora based on four independent controversial issues. These corpora are relatively small, they are designed to explore the needs in terms of knowledge, knowledge organization and reasoning schemas. The goal is to elaborate the main features of a larger empirical analysis and functional model.

For this first experiment, we considered the four following issues, which involve very different types of arguments, forms of knowledge (concrete or relatively abstract) and language realizations. These issues are:

Texts have been collected on the web, considering the issues as web queries. The text fragments which have been selected are extracts from various sources where these issues are discussed, in particular: newspaper articles and blogs from associations. These are documents accessible to a large public, with no professional consideration, they can therefore be understood by almost every reader. The knowledge that is involved to understand them is relatively simple and is shared by most of the readers. Language is French (glosses are given in the paper) or English. Sources are, for example:

A large number of texts have been collected for each issue. For each text, the next task was to manually identify arguments related to the issue at stake. The unit considered for arguments is the sentence. These sentences are then tagged ⟨argument⟩. Since texts may be quite verbose, may include factual data of no argumentative interest, have pictures or data that cannot be taken into account in this investigation, some cleaning is necessary. In this investigation, only the paragraphs that containt a sentence identified as an argument are kept.

A preliminary task aimed at identifying all the arguments related to each issue. In a second stage, arguments judged by the annotator to be similar or redundant are eliminated and a single utterance, judged to be the most representative in terms of content and structure, is kept. The arguments eliminated on the redundancy criteria are kept separately for further analysis or for tests. So far, due to a lack of human resources, these tasks have been realized only by the author. It is clear that decisions made are somewhat subjective and that at least two annotators would have been necessary. In spite of these limitations, we consider that this work is a valid preliminary investigation whose main aim is (1) to gradually develop relevant annotation guidelines, as it is the case for most argument annotation projects, (2) to evaluate the impact of knowledge in argument mining processes and (3) to suggest new investigation directions. A more elaborated analysis must follow with the involvement of several trained annotators, with the progressive development of clear and stable annotation guidelines.

For each of these issues, the corpus characteristics and the different arguments found are summarized in Table 1. In this table, the number of words in column 2 represents the total size of the text portions that have been considered for this task, i.e. those paragraphs that contain at least one argument.

This corpus shows that the argument diversity per issue is not very large. A relatively high overlap rate has been observed: while there are original arguments, authors tend to borrow quite a lot of material from each other. For example, for issue (1) an average redundancy rate of 4.7 has been observed, i.e. the same argument is found 4.7 times on average in different texts. This overlap rate is somewhat subjective since it depends on the annotator analysis and the corpus size. In spite of this subjectivity, this rate gives an interesting rough redundancy level. It is probable that with a larger corpus, this overlap rate would increase, while the number of new arguments would gradually decrease. A more detailed analysis of those repetitions would be of much interest from a rhetorical and sociological perspective.

A last step in the corpus analysis consists in tagging the discourse structures found in those sentences identified as arguments. For that purpose, the TextCoop platform we developed [17] is used with an accuracy of about 90%, since sentences are relatively simple. Discourse structures which are identified are those usually found associated with arguments: conditions, circumstances, causes, goal and purpose expressions, contrasts and concessions. The goal is to identify the kernel of the argument (tagged ⟨mainarg⟩), in general the main proposition of the sentence, and its sentential modifiers. In addition, the discourse structures may give useful indications on the argumentation strategy that is used. The result of the tagging, at this stage, is e.g.:

3.2.Annotation parameters

The next step is to define a set of preliminary tags appropriate for analyzing the impact and the types of knowledge involved in argument mining. In this annotation task, our approach is twofold:

An argument and its context (the discourse structures it is associated with, such as an illustration or a condition) are tagged between XML ⟨argument⟩ tags. The argument kernel is tagged ⟨mainarg⟩. Attributes, associated with the main tag ⟨argument⟩, informally characterize the knowledge that is required, they include:

A argument from issue (1) is tagged as follows:

This example is a rather weak attack of issue (1) since it says that more conclusive data is necessary before making large vaccination campaigns. From that point of view, this argument may be analyzed and labeled as a concession: vaccination is necessary, however it is necessary to wait for more conclusive data.

The annotation task is realized in several steps. The specification and the language expressions used in the two attributes ‘Conceptual relation to the issue’ and ‘Concepts involved’ are proof-read and polished several times in order to have homogeneous specifications for each issue and over the four issues which are developed in this investigation. Concepts used are kept as minimal as possible, while the most accurate level of granularity is preserved from our intuitions. After this stage, a conceptual analysis of how arguments are related to an issue is carried out. This is developed in the next section.

4.1.Main concepts in arguments related to issues (1) and (4)

In issues (1) and (4), arguments mainly attack or support salient features of the main concepts of the issue or closely related ones by means of various forms of evaluative expressions. Samples of arguments associated with issue (1) are:

For issue (1), the term vaccine is the central concept of the issue. The concepts used to express arguments for or against that issue can be structured as follows, from this central concept vaccine:

For example, the concepts of side-effect and toxicity are consequences of vaccines and more generally of a medicine, which, for a large number of them may have side-effects, besides curing a given disease. The concept of contamination is related to one of the purposes of a vaccine, namely to avoid that other people get the disease via contamination, and therefore on a larger scale, the purpose of a vaccine is to prevent disease dissemination. Finally, production costs are related to the creation and development of any product, including medicines and vaccines.

Without knowing that a vaccine protects humans from getting a disease, it is not possible e.g. to say that prevents high contamination risks is a support for issue (1) and to explain why. Similarly, without knowing that the active principle of a vaccine is diluted into an adjuvant that is also injected, it is not possible to analyze the adjuvant is toxic as an attack. Without this knowledge, this statement could be e.g. purely neutral or irrelevant to the issue.

The conceptual categories used in this short analysis: purpose, functions, goals, properties, creation and development, etc. are foundational aspects of the structure of a concept. They allow an accurate identification of arguments and what facet they exactly attack or support in the issue and how. This is also useful to construct various types of argument synthesis.

4.2.Main concepts in arguments related to issue (2)

This issue is much more complex and abstract than the issues (1) and (4); it is useful to evaluate the boundaries of our approach and its feasibility. The arguments related to issue (2) mainly involve comparisons with men’s living conditions or refer to general and quite diverse principles of human welfare. Some arguments are justified by means of figures while others remain vague, possibly not up-to-date or relative to specific situations. Samples of arguments found for issue (2) are:

The concepts used in arguments related to issue (2) concentrate on facets of humans in the society and in the family and evaluates them for women. The identification of these concepts is crucial to characterize argumentative relations between issue (2) and their polarity. For example, improving literacy means higher education, better jobs and therefore more independence and social recognition, which are typical of a living condition improvement.

Roughly, the concepts used in arguments supporting or attacking issue (2) can be classified into two categories:

4.3.Main concepts in arguments related to issue (3)

In the case of issue (3) (nuclear plants are necessary), supports or attacks mainly involve comparisons between various sources of energy. Samples of arguments are:

In terms of concept and concept organization, nuclear plants are complex entities. A first set of concepts concern their creation, i.e. construction, development and maintenance, a second set concerns their purposes, functions, and consequences which can be further decomposed into facets. Parts are also addressed, for example in the case of nuclear wastes. The same structures must be developed for other sources of energy (coal, water, wind, etc.) to analyze those arguments which include comparisons between nuclear energy and other sources of energy.

Most arguments lack precise comparative data, e.g. wastes are all said to be a nuisance, but the degree of nuisance is not given. These arguments rather play on the pathos. These are underspecified arguments. Finally, some arguments are fallacious, for example, the ‘energetic independence’ advocated as a support for nuclear plants is not acceptable since uranium is bought abroad: the use of knowledge should contribute to detect them.

4.4.Observation synthesis

From the above manual analysis, the following argument polarities are observed:

Arguments where the need of knowledge between an issue and an argument has been characterized is summarized in Table 2. For example, for issue (1), 44 arguments for a total of 50 (88%) require knowledge to be identified as arguments. For these 44 arguments, 54 different concepts are required to establish that these statements are arguments for or against issue (1).

From the annotation schemes (Section 3) and the analysis provided in the above Sections 4.1 to 4.3, it is possible to identify the main types of knowledge which are involved. From our corpus observations, it turns out that the types of knowledge involved in relating an argument to an issue are based on the existence of lexical semantics relations between the concepts of the main terms of the issue and the argument. These relations typically include:

Paradigmatic and the specific functional relations presented above cover a large majority of the relations between issues and their related arguments estimated to about 80% of the total. There are obviously other types of relations, more difficult to characterize, e.g. vaccination prevents bio-terrorism, but of much interest for argument mining.

The figures in Table 2 show that for about 77% of the statements identified as arguments, some form of knowledge is involved to establish an argumentative relation with a controversial issue. An important result is that the number of concepts involved is not very large: 124 concepts for 100 arguments over 4 domains. Even if the notion of concept remains vague, these results are nevertheless interesting to develop large argument mining systems.

Argument behavior can be summarized as follows:

The above analysis shows that the introduction of knowledge in argument mining is a crucial feature which allows to improve the identification of:

4.5.Language realizations

In terms of language realizations, the following three main types of argument realizations are observed:

Based on the notion of evaluative expression in (1) above, the following constructions are frequently observed:

5.1.Perspective and position

The conceptual organization described in the above analysis tends to suggest that the type of conceptual categorization offered by the Qualia structure, in particular in its last development in Generative Lexicon (GL) [20], with some extensions to the formalism, is an adequate representation framework to deal with knowledge based argument mining. The remainder of this article illustrates this proposal that needs to be evaluated in more depth when the annotation phase described above becomes more accurate and confirmed by several annotators.

The main other lexical semantics approaches such as FrameNet or VerbNet mainly concentrate on predicate argument structure for verbs and their adjuncts: they characterize the roles that these elements (NPs, PPs and S) play in the verb and proposition meaning. According to our observations, these are not central features for knowledge-based argument mining although they may be useful to develop lexical resources and argument mining templates as shown in [19].

The GL is a model that organizes concept descriptions via structures called roles. Roles describe the purposes and goals of an object or an action, its origin, and its uses. These are the main features which are supported or attacked in arguments. These, to the best of our knowledge, are specific features of the GL Qualia structure, in particular the telic and the agentive roles. However, a main limitation is however that the GL has very little resources available.

The notion of Qualia structure emerged from Aristotle categories. It has been much debated over the centuries in epistemology and philosophy. Contradictory views and definitions abound which will not be addressed in this article. Our perspective is that some features of Qualias developed in the GL seems to be appropriate to represent the type of knowledge that is crucial in knowledge-based argument mining. A number of features and uses of the Qualia (e.g. as supports of type coercion to represent logical metonymy) are not relevant for our investigations. The Qualia is therefore a convenient formalism and a set of tags that does correspond to argument mining needs.

In the remainder of this section, after a short presentation of the Qualia structure of the GL, we develop a few motivational examples, that make more concrete the proposals developed in Section 4. To conclude, some insights on the potential diversity of arguments associated to an issue are developed with an outline of a model that partly characterizes how Qualias can be used to account for the generative expansion of arguments. Our experience is that while there are a few unexpected arguments, most of them can be relatively well circumvented on the basis of the concepts developed in Qualias.

5.2.An introduction to the Generative Lexicon

The Generative Lexicon (GL) [20] emerged from a series of research efforts inspired by Aristotle’s notion of modes of explanation. The GL is an attempt to structure lexical semantics knowledge in conjunction with domain knowledge. It allows to explain a number of language phenomena such as various types of metonymies via a decompositional view of lexical meaning. Various forms of so-called ‘generative aspects of lexical combinations’ have been characterized via the operation of type shifting, where the original type that is expected has been coerced to another type, allowing metaphors such as ‘to devour books’ or various forms of sense variations, which are frequent in language. The GL introduces some original forms of representations such as dotted types, that account for the different facets of an entity (e.g. the physical and contents facets of a book). The GL also develops a specific argument structure with semantic types, lexical paradigms, an event structure and the Qualia structure, which is the structure that is considered in our investigations.

The Qualia structure of an entity is a kind of lexical and knowledge repository composed of four fields called roles:

Roles are composed of constants or predicates a priori defined from a domain or from a general purpose ontology. A well-known example is the case of novel(X):

In this Qualia structure, X is the novel, Y, a reader, T, the author, and P, the publisher. Concepts are represented by means of predicates or constants. In the constitutive and agentive roles, two ‘facets’ of novel are encoded: the physical object and the contents. In the telic role, the prototypical predicate is read; the other predicates such as inform, entertain, develop additional purposes. In the agentive role, the creation of a book is characterized in particular by its writing (content facet), printing and publishing (physical object facet). Predicates such as ‘sell’ may belong to the two facets.

The formal role is close to the isa (hyponymy) relation frequently found in ontologies and in WordNet. The constitutive role has a rich informational structure since the different types of parts or constituents can be specified in dedicated sub-roles. The argument structure of the GL and its semantic typing is different from frame semantics descriptions such as FrameNet which basically develops semantic roles, not present in the GL. We show below that these two roles structure most of the knowledge that is required for a knowledge-driven approach to argument mining.

5.3.A Qualia representation of the issue head concepts

This section shows how the concepts informally presented in Sections 4.1 to 4.3 can be represented in Qualias and what extensions are required. The goal is to show how concepts can be specified in a concrete manner. This section introduces (1) the extensions to the GL that are required for argument mining and (2) the elements of a method to describe the type of data that is needed for knowledge driven argument mining. The Qualias are a relatively open representations where roles may be more or less developed and constrained: a method to describe Qualia roles is crucial to develop homogeneous and coherent sets of Qualias. Some role sub-typing have been introduced in the past in the EEC Parole-Simple project (http://catalog.elra.info/product_info.php?products_id=881&language=en).

The Qualia structure of the head term of issue (1), vaccine, describes the structure and the main properties and functions of a vaccine: where X is the variable that represents the vaccine, Y is the person that is vaccinated, T is the biologist or company that develops the vaccine, Z is the doctor that makes the injection, and D is the disease associated with the vaccine (Ebola in our example). The agentive role develops the way the vaccine is created while the telic role develops its functions and purposes. An example of typing is introduced in this role: main functions (protect, avoid dissemination) and the means: how these functions are realized. This Qualia representation can be extended to develop causal and temporal chains: where E1, E2 and E3 are event-denoting variables.

This Qualia is written manually and the concepts it contains are derived from the arguments found in texts. Additional purposes or functions can be found using e.g. bootstrapping methods on the web via patterns such as ‘a vaccine allows to’. If quite a lot of data can be acquired automatically, we feel that some parts of the overall organization needs to be done manually, as in most knowledge representation systems, to guarantee coherent and well-structured representations.

Next, the Qualia structure of Ebola (and more generally, of a virus) is: P represents the patient that gets the disease. The purpose of Ebola is to infect people (P) who get sick and may die. There is no agentive role since there is no volition in the Ebola virus. Ebola is a constant in this representation. To characterize the sequence of events, a sequence of causal chains from ‘infect’ to ‘may die’ is introduced in the telic role.

The Qualia associated with the head concept in issue (2) is more complex and has several facets. The Qualia represents the facets of humans in the society and at home. The Qualia represents the functions, roles and purposes of humans in the society and at home. Women’s conditions is evaluated w.r.t. these features. This Qualia includes the main roles of humans as found in ontologies or in texts via mining. They are organized via several informal types such as: socio-economic, society, family, etc.:

The agentive role develops the emergence of the citizen and the parent, e.g. via education. The telic describes the main roles humans play in these environments. This example, abstract and very generic, shows the limits of Qualias and also probably of most knowledge representation systems (Cyc, Sumo, etc.). However the level of granularity shown here seems to be sufficient for the purpose of argument mining.

For issue (3) the head term is the compound ‘nuclear plant’. Its Qualia can be summarized as follows:

The events Ei are subject to partial temporal constraints, in particular: E1, E2, E3, E4 < E5, E9, E7, E8. There is a temporal overlap between the agentive and the telic. For plants producing electricity from coal, wind or water, the Qualia is almost identical, with the difference that uranium must be replaced by either coal, wind or water. The control organizations are also different, probably government institutions. Wastes or nuisances must also be specified. Given these conceptual similarities, the comparisons found in arguments can be related to the controversial issue.

Describing Qualias raises several feasibility problems since there is little data available. The problem of an automatic or semi-automatic acquisition of Qualias is addressed in e.g. [21], and to a lesser extent in [19]. While it is possible to encode Qualias manually, we feel that a bootstrapping method from corpora should enhance the process by providing a lot of data, which, however, needs to be organized by hand in a second stage. An interesting point is that, for a given issue, the knowledge that is required is not so large, as shown in Table 2. It seems that for an issue with a large scope, a maximum of 20 Qualias are needed to deal with most of the arguments. For the issues presented in this article, the number of required Qualias is given in Table 3, for the arguments that have been identified.

The number of Qualias for an issue in the corpus presented in Section 3 is on average 10. Some Qualias may be quite large while others are relatively sketchy. Some Qualias may be close to or inherit from others, e.g. vaccine and medicine. The number of Qualias may slightly change depending on how knowledge is structured, but the variation is quite limited and the number of concepts to include in the Qualias remains approximately constant. Table 3 shows that an average of 4 concepts are used per Qualia in our corpus.

5.4.A network of Qualias to characterize the generative expansion of arguments

Our observations show that arguments attack or support (1) specific concepts found in the Qualia of the head terms in the controversial issue (called root concepts) or (2) concepts derived from these root concepts, via their Qualia. For example, arguments can attack properties or purposes of the adjuvant or of the protocols used to test the vaccine. Then, a network of Qualias can be defined to develop the knowledge structures involved in the recognition of potential arguments [19]. This network is limited to three levels because derived concepts must remain conceptually and functionally close to the root concepts. The objective is to evaluate a kind of ‘transitive closure’ that would characterize the typical and most frequent concepts that appear in arguments that support or attack an issue. Obviously, unexpected arguments may arise, probably with a lower frequency and recurrence.

This network of Qualias forms the backbone of the argument mining system. This network develops the argumentative generative expansion of the controversial issue. This network is also the organization principle, expressed in terms of relatedness, that guides the generation of a synthesis where the different facets of the Qualias it contains are the structuring principles [13]. Natural language words or expressions that lexicalize each concepts can be associated with each network nodes.