Semantic clause types and modality as features for argument analysis¹

3.1.Situation entity types

To investigate the semantic types of clauses found in argumentative text passages, we use the inventory developed by [30] and extended in later work [8,22]. Situation entity (SE) types describe the semantic types of situations (including states, events, generics, habituals, questions, and imperatives) evoked in discourse by individual clauses of text. As such, they concern the manner of presentation of content along with the conveyed information.

SE types are recognizable (and distinguishable) through a combination of linguistic features of the clause and its main verb, and it was found that the distribution of SE types in text passages correlates to some extent with whether the text passage is (e.g.) narrative, informative, or argumentative [3,17,21].

The use of linguistic features for distinguishing text passages is closely related to Argumentative Zoning [20,32], where linguistic features are used to distinguish genre-specific types of text passages in scientific texts. In this manner, those texts are segmented into types of text passages such as Methods or Results. Notions related to SE type have been widely studied in theoretical linguistics [1,4,6,29,33,34] and have seen growing interest in computational linguistics [5,11,12,19,25,28,35].

Annotation categories. The inventory of SE types considered in our analysis starts with states and events (cf. Table 1 and Fig. 2 for examples from the microtexts).

Modality, negation, future tense, and conditionals, when coupled with an Event-type clause, cause a coercion to State. In these constructions, reference is made to actual or potential states of the world rather than to actual events that have occurred. Several examples appear below; note that no coercion happens for State clauses.

The other two frequently-occurring SE categories in argumentative texts are generic sentences and generalizing sentences, the latter sometimes referred to as habituals. While the former predicate over classes or kinds, the latter describe regularly-occurring events, such as habits of individuals.

The next category of SE types is broadly referred to as Abstract Entities. This type of clause presents semantic content in a manner that draws attention to its epistemic status. We focus primarily on a small subset of constructions – factive and propositional predicates with clausal complements. Of course a wide range of linguistic constructions can be used to convey such information, and to address them all would require a comprehensive treatment of subjective language. In the examples below, the matrix clause is in both cases a State, and the embedded clause (in italics) is both an Event and either a Fact, Proposition, or Resemblance-type situation entity. Note that in the Resemblance case, it is unclear whether or not the embedded event took place.

Finally, the labels Question and Imperative are added to allow complete annotation of texts.

Annotators had access to the full inventory of ten SE types described above, but only four of them occur often enough in the microtext corpus to allow for meaningful analysis (see Section 4 for details).

3.2.Modal verbs and modal senses

Modal verbs convey extra-propositional meaning of clauses, encoding information about possibility, necessity, obligation, permission, wishes, or requests [13]. Features of the verb such as modal auxiliaries, tense, and mood have been widely used in previous work for classifying argumentative vs. non-argumentative sentences [7,18]. For example, [18] include verb lemmas and modal auxiliaries as features, and [7] find that, for Greek web texts related to public policy issues, tense and mood features of the verbal constructions are helpful for determining the role of the sentences within argumentative structures. Classifiers that automatically distinguish modal senses in context have been developed by [26] and improved in subsequent work [15,16,36].

We annotate all modal verb occurrences in the microtexts with their modal sense, following the inventory of modal senses and annotation guidelines from [36]. They distinguish three senses: epistemic (ep), indicating possibility/necessity, deontic/bouletic (de), indicating permission/request/wish, and dynamic (dy), indicating ability. Different types of modalities express different types of extrapropositional meaning: dynamic modality captures knowledge about specific capacities of certain entities or types of entities, while epistemic modality expresses knowledge about how likely a certain proposition is. Deontic modality expresses wishes or requests that are desired or undesired, and that may be possible or counterfactual. We illustrate these modal senses below with sentences taken from the microtext corpus:

It is important to mention that the interpretation of modal verbs can vary with their context. [36] find that specific sense ambiguities such as dynamic vs. deontic readings of can, epistemic vs. dynamic readings of could, or epistemic vs. deontic readings of should are difficult to discriminate. Likewise, senses can be expressed through different modal verbs. In the following example, a possibility (epistemic sense) is expressed by two different German modal verbs (mögen and können):

3.3.Extending argumentative annotations of microtexts to SE types and modal senses

We annotate all segments with their situation entity types, and those containing modal verbs, with their modal senses. Annotation examples are illustrated in Table 1 and Fig. 2 which also give examples of both SE types and modal verbs and their senses in argumentative texts. The premise Sure, other people have to work in the shops on the weekend (Table 1) is an example for a Generic Sentence, while the premise This should be clear to everyone after the disclosures of Edward Snowden in Fig. 2 is a State and the last segment in Fig. 2 (Their tools, data and expertise have been used to keep us under surveillance for a long time) a Generalizing Sentence.

Segmentation. Before annotating the microtexts, we first segment them into clauses (cf. [3]). As seen in Table 1, an argumentative component can in fact consist of several SE segments. The granularity of situation-evoking clauses is different from that of argument components, requiring that the texts be re-segmented prior to SE annotation. For segmentation we use DiscourseSegmenter [27], a Python package offering both rule-based and machine-learning based discourse segmenters.44 After preprocessing texts with the Mate Tools,55 we use DiscourseSegmenter’s rule-based segmenter (edseg), which employs German-specific rules to determine the boundaries of elementary discourse units in texts. Because DiscourseSegmenter occasionally oversplit segments, we did a small amount of post-processing. On average, one argument component contains 1.16 clauses.66 These clauses are then annotated with SE types and modal verbs and senses.

Fig. 2.

Argumentation graph with clause type and modal sense annotations.

Agreement statistics. We train two student annotators with backgrounds in linguistics for labeling texts with both SE types and modal senses. The two annotators then label the microtexts independently. We compute agreement both for SE types and for modal senses and gain an inter-annotator agreement of 0.40 for the former and 0.75 for the latter, both reported as Cohen’s Kappa. As reported by several studies [23], the annotation of argumentative texts is a difficult task for humans. The agreement statistics reported above suggest that this is also the case for the task of labeling argumentative texts with SE types. The SE-labels that caused most disagreement among the annotators are State vs. Generic Sentence (39% of all disagreements), followed by State vs. Generalizing Sentence (22%). For modal verbs, most disagreements are caused by the labels dynamic vs. epistemic (60% of all disagreements), followed by epistemic vs. deontic (24%). We then produce a gold standard annotation by re-annotating all segments for which the two student annotators disagreed about the SE type and/or the modal sense. This third annotation is done by an expert annotator (one of the authors), and we use the adjudicated annotations for our analysis.

4.1.Distributions of SE types within different genres

We find that the most important SE types within the microtexts are State, Event, Generic Sentence, and Generalizing Sentence, while the remaining six types occurred not at all or infrequently: Question (10 segments), Proposition (2 segments), Resemblance (2 segments), Fact (0 segments), Report (0 segments), and Imperative (0 segments). Thus, our further analysis will focus only on the four most frequent SE types State, Event, Generic Sentence, and Generalizing Sentence.

In order to investigate whether argumentative texts can be distinguished from non-argumentative texts with respect to the distribution of SE types (cf. [30]), we compare the distribution of States, Events, Generic Sentences, and Generalizing Sentences in the microtext subcorpus to those of other text genres. Therefore, we first segment and then annotate a small corpus with texts of different genres with SE types. This corpus includes the following genres: fiction (47 segments), reports (42 segments), TED talks (50 segments) and commentary (127 segments).

The microtexts, which can be described as “purely” argumentative texts, are characterized by a high proportion of Generic Sentences and Generalizing Sentences and very few Events, while reports and talks, for example, contain a high proportion of States. This is displayed in Fig. 3. Fiction is characterized by a high number of Events. The commentary genre, which contains many argumentative passages but is not as purely argumentative as the microtexts, is most likely to be comparable to the microtexts. This finding suggests that SE types could be helpful for modeling argumentative regions of text.

Fig. 3.

Distribution of SE types within different genres.

4.2.Correlations between SE types and argument components

Generic Sentences are the most frequent type overall, distinguishing argumentative texts from other genres, in which Generic Sentences occur quite infrequently [8]. In the microtexts, conclusions are almost exclusively Generic Sentence or State, while premises, next to a majority of Generic Sentence, include a higher proportion of Generalizing Sentences and Events (cf. Table 2 and Fig. 4(a)). This is in line with the role of the conclusion in the argument – it should concisely state the supported position, which is more likely to be a statement about a class of things (e.g. Not everyone should be obliged to pay TV license) or a condition or attribute of the world (e.g. Raising the retirement age on the basis of physical fitness of the average citizen is fair) rather than reference to an episodic event.

Looking at premises only, Generic Sentences are extremely frequent for proponent premises (52%) and less so for opponent premises (42%) (cf. Fig. 4(b)).

Fig. 4.

SE types and argument components and propositions.

Table 2 shows the distributions over the four major SE types for the various types of argument components in the microtext corpus.

4.3.Correlations between SE types and argumentative functions

Turning to correlations between SE types and argumentative functions of premises, we focus on the three most frequent functions: support, rebuttal, and undercut (Table 2, rightmost section, and Fig. 5). Supporting premises have a SE type distribution very similar to that of proponent premises overall. This is not surprising given that most microtexts contain only a single opponent premise. Undercutting premises show an even higher frequency of Generic Sentences (56%), with the caveat that undercuts occur less frequently than the other types. Finally, rebutting premises show a lower frequency of Generic Sentences (only 42%) and a fairly high occurrence of States (32%). This tendency can be illustrated with the following example, where a State (bold face) rebuts the conclusion (in italics).

Fig. 5.

Correlations between SE types and argumentative functions.

These results suggest that situation entity types could be helpful even for a finer-grained analysis of argumentative functions.

Of course, our annotated corpus is very small, and the phenomena we observe can be interpreted solely as tendencies. Nevertheless, we hypothesize that the prevalence of the Generic Sentence and Generalizing Sentence labels, as well as the absence (or rareness) of the Event label are indicators not only of argumentative text passages, but in particular of conclusions and proponent argument components.

4.4.Analysis on the dataset without considering coercions

As mentioned above, linguistic features such as modality, negation, future tense or conditionals cause a coercion from Event-type to State. We find 84 such coercions in our dataset, 30 within conclusions and 54 within premises.

To better understand the effect of these coercions on our analyses, we reiterated our analyses on the same dataset, but undoing these coercions. This means that we use a version of the dataset in which what ordinarily would be treated as cases of coercion are instead labeled with the SE type that holds prior to coercion. Example (26) e.g. would be labeled as Event instead of as State:

The distributions of SE types within the dataset with and without coercions can be found in Table 2 (version without coercions in brackets) and are visualized in Fig. 6.77 Since coercion affects the proportion of State- and Event-types, we of course find a smaller number of States and a higher number of Events in this analysis. As Fig. 6 shows, the relation between States and Events changes the most for conclusions. Instead of only 3% Events (when applying coercion), we find 30% Events (without coercion), i.e., a difference of 27 percentage points (for premises, the difference is only 12 percentage points). Moreover, while in the analysis considering coercions we find a higher percentage of States compared to Events, in the analysis ignoring coercion the reverse is true. Here we find more Events than States for all categories (argument components as well as argumentative functions) – except in supporting premises, where we find 19% States and only 18% Events.

Fig. 6.

Distributions of SE types within argument components and argumentative functions, with and without coercions.

This supports the assumption that episodic events are dispreferred for supporting a conclusion or a premise, but preferred for attacking a conclusion or another premise. This second claim is supported by the high proportion of Events within rebutting and undercutting premises (28% and 25%). In other words, anecdotal reference to events occurs more often for attacking than for supporting. This can be illustrated with the following example, where a generic premise (in italics) is rebutted by a negated and modalized premise which is accordingly coerced from Event to State:

Our motivation of comparing the datasets with and without coercions was to learn more about the effect that modals and other linguistic features which cause coercions may have on the distribution of SE types within argumentative texts. We find that indeed there is a considerable effect: as reported above, we naturally find fewer Events within the dataset without coercions – a distributional effect that one can possibly exploit in an automatic system. What we also find is that 59 of the 84 coercions (70%) in our data are due to modal verbs. We thus complement our study of SE types with an investigation of the role of modal verbs and their senses within argumentative texts.

5.1.Modal verbs within argumentative and non-argumentative texts

In our annotation we found that modal verbs constitute a high-frequency linguistic phenomenon in the microtexts. The overall 188 modal verbs included in the microtexts occur in 31% of all argumentative segments (57% of conclusions and 26% of premises) and are thus at least partly responsible for the high number of States, due to the coercion of Events to States when embedded under a modal verb. In comparison, only 5% of the segments of the mixed-genre subcorpus introduced in Section 4, including genres such as fiction, report, commentary and TED talks (see [3]), contain modal verbs. Due to this coercion, the contribution of modality indicators is orthogonal to SE types. We thus complement the analysis of SE types with a deeper investigation of modality-indicating expressions, here modal verbs.

Table 3 shows the German modal verbs and their available senses. Note that wollen is not a modal verb, but was included because mögen (may/want to) is ambiguous, and thus we included the synonymous wollen and tagged this sense as deontic (wish).

Our analysis looks closely at distributions of both modal verbs and modal senses and is performed on a subset of the segments from the SE type analysis, as we are concerned only with segments containing modal verbs (cf. first row in Table 4).

5.2.Correlations between modal verbs/senses and argument components

Overall, the most frequent modal verb in the microtexts is sollen (121 occurrences, 41% of modal verbs), and the most frequent sense across all modal verbs is deontic (133 occurrences, 75%).

Fig. 7.

Modal senses and argument components and propositions.

Of the conclusions that contain a modal verb, an overwhelming 97% (62/64) are deontic; compare this to premises that contain modal verbs, for which only 61% (71/117) are deontic (Fig. 7(a)).

The prevalence of sollen within conclusions (86%) warrants a caveat: most of the trigger questions contain some form of sollen, and so it is possible that the predominance of sollen, particularly in conclusions (86%), is a result of the form of the trigger questions.88 However, the distributions of modal verbs within the subset of microtexts not associated with a trigger question are very similar to the overall distributions (among conclusions: sollen = 77%, können = 8%, müssen = 15%). This suggests that the observed distributions are not overly influenced by the trigger questions.

The other nearly 40% of premises are epistemic and dynamic senses of MVs, which occur in conclusions with a combined frequency of only 3%. The prevalence of these two senses is reflected in the high frequency of können, which frequently occurs as dynamic or epistemic. As is generally the case, müssen in epistemic sense is rather rare. The tendencies can be seen clearly in the upper part of Table 4, which displays the modal sense distributions.

In premises, the most frequent modal verb is können, at 36%, followed by müssen with 25%. These again stand in stark contrast to conclusions, where these two verbs occur with frequencies of 3% and 7%, respectively. A similar observation concerns proponent and opponent premises (Fig. 7(b)). Here, 64% of proponent components with a modal verb are deontic, while this is true only for 53% of opponent components with modal verbs. Instead, we find a relatively high proportion of epistemic senses (32%) for opponent premises, and a similarly high proportion for epistemic sense is found with rebutting premises.

5.3.Correlations between modal verbs/senses and argumentative functions

Epistemic modals refer to alternative, often counterfactual worlds, and as such they are well suited for rebutting preceding or hypothesized arguments or premises (Fig. 8). An example of this from our data is given in example (28), where bold face marks a rebutting premise:

Fig. 8.

Correlations between modal senses and argumentative functions.

Turning to dynamic sense, we find it more strongly associated with supporting function. We typically find this sense realized with können, bringing forth abilities or capabilities of entities, presented as options and chances, in support of a premise (29).

Finally, even though deontic is the most frequent modal sense within all of the microtexts, it is especially frequent within undercuts: 76% of the undercuts that contain modal verbs have the modal sense deontic (obligation).

As an example, we find (30):99

In rebuttals with modal verbs (30 occurrences) we count only 50% deontic and 39% epistemic modal senses, see (31) for a deontic rebuttal.