RelTopic: A graph-based semantic relatedness measure in topic ontologies and its applicability for topic labeling of old press articles

1.Introduction

Fig. 1.

Excerpt of Le Matin.

This article presents the works accomplished as part of the ASTURIAS11 project in the domain of cultural heritage. The main goal of ASTURIAS is to thematically and automatically organize a collection of old press articles with a set of topics (e.g., Politics, Art, Sport, Science, Etc.). One of the specific features of old press is that it does not offer thematic entries (see Fig. 1). Articles appear and follow one another without a thematic logic. Under these conditions, it remains a tedious task to query sources that report the same events from different points of view in different areas of the newspaper. The scientific challenge is to propose robust approaches for the analysis of texts that are noisy due to the imperfect process of automatic transcription of images into electronic texts. These approaches need also to be multi-thematic, and robust to linguistic evolution over the centuries. The ambition of the ASTURIAS project (whose workflow appears in Fig. 2) is to study the digitization process from end to end of the processing chain: WP1 – from newspaper images, automatically analyze sections, articles and texts; WP2 – extract named entities from these elements; WP3 – Topic labeling and hyperlinking the articles based on the analysis made in WP1 and the named entities extracted in WP2.

Fig. 2.

The pipeline of the project ASTURIAS.

Our work’s main goal is to propose a framework that permits automatic labeling of old press articles (WP3). This framework tends to replace humans with software for labeling a vast number of articles that would require too much human effort to do it manually. The task of labeling documents according to their topics has traditionally been addressed either by using classifiers for assigning to the articles a set of predefined topics (e.g., [8,39]), or by topic detection methods (e.g., probabilistic latent semantic analysis (pLSA) [22], latent Dirichlet allocation (LDA) [2]), which generate topics from textual resources [39]. The main advantage of the first approach is generating clean, and formally-defined research topics [39]. This approach is recommended when a good characterization of the research topics within a domain is available [39]. However, the second approach suffers from a significant limitation of generating topics from scratch leading to noisier and less interpretable results [39].

In this study, we propose applying graph-based semantic relatedness measures that permit assessment of the semantic relatedness of topics in topic ontologies with articles’ content. Graph-based semantic measures have been used to solve problems in a broad range of domains such as Natural Language Processing (e.g., [16]), Information Retrieval (e.g., [17]), Knowledge Engineering (e.g., [14]), Semantic Web and Linked Data (e.g., [9]) and Bioinformatics (e.g., [18]). They are considered essential tools for designing numerous algorithms in which semantics matters [19]. A graph-based semantic measure is a mathematical tool used to estimate the strength of the semantic interaction between entities (concepts or instances) based on the analysis of ontologies [19]. Thus, the application of this measure is strongly dependent on the availability of an ontology that represents the application domain. Two main categories of graph-based semantic measures are distinguished: (1) similarity measures adapted to taxonomies and (2) relatedness measures adapted to semantic graphs composed of different types of relationships [19]. Building semantic relatedness measures is a challenging and important research issue since they have to consider several kinds of relations and not only the taxonomic ones [30]. In the literature, apart from Hirst and St-Onge’s measure [21], there have been relatively few attempts to develop relatedness measures [30,34]. Most efforts are directed to design similarity measures such as [25,26,35,37]. For comparing ontological entities, graph-based measures are classified into two basic approaches: path-based, which compare the concepts according to properties of paths in graphs, and node-based, that use properties of concepts in the ontology graph for comparing concepts. However, these approaches suffer from different limitations.

The major contribution of this work is the design and evaluation of a semantic relatedness measure, named RelTopic, that considers the semantic properties of entities in topic ontologies. RelTopic is designed as a combination of node-based and path-based approaches. In contrast to existing measures, our measure tends to assess the relatedness of concepts and instances by considering different types of relations. RelTopic will be used for topic labeling of old press articles, which are represented by a set of “not ambiguous” named entities extracted from open data sources (WP2). A second contribution to mention is building a topic ontology named Topic-OPA from the open knowledge graph Wikidata using a SPARQL-based automatic approach. Topic-OPA is required for the application of RelTopic. Based on RelTopic and Topic-OPA, we defined the selection process of the most relevant topics for labeling the articles. To demonstrate the performance of our approach, a use-case is presented in the context of Le Matin,22 an old French newspaper first published in 1884 and discontinued in 1944. Finally, Topic-OPA and RelTopic are evaluated using dual evaluation approaches. Our approach’s reusability is demonstrated for labeling articles in different contexts, such as recent newspapers. The implementation of our study is available in GitHub33 [31].

The remainder of this paper is organized as follows: the research problem is specified in Section 2. Section 3 considers the main related works. In Section 4, we discuss our semantic relatedness measure RelTopic. Section 5 introduces Topic-OPA. Section 6 discusses the topic labeling process. In Section 7, we present a use case for labeling the articles of Le Matin. We evaluate and discuss the approach in Section 8 and Section 9 respectively. Finally, Section 10 concludes the paper.

2.Problem definition

To define our research problem, a fundamental hypothesis is considered that articles are represented by a set of “not ambiguous” named entities (e.g. person, organization, product and location) extracted from open data sources (coming from WP2 of the ASTURIAS project, as shown in Fig. 2). Thus, the research problem can be defined as follows: given a corpus of articles A, a set of named entities N (represented by a set of URIs) that are collected from A (WP2), and a topical structure T, we want to find the most relevant topics described in T that label Ai, ∀Ai∈A. Based on this perspective, our work (WP3) considers mainly the following issues:

3.Related works

This section outlines the following related works: graph-based semantic measures, semantic relatedness measures, topic ontologies, and ontology-based labeling of articles.

4.Our semantic relatedness measure

In this section, we propose a hybrid graph-based semantic relatedness measure within topic ontologies. As a contribution to the community of approaches that tend to overcome the limitations of existing measures (e.g., [30]), we designed our measure as a combination of path-based, and node-based approaches. Thus, we comprehensively consider the semantic properties of nodes and edges:

5.Topic-OPA: A topic ontology for modeling topics of old press articles

In this study, RelTopic is applied within topic ontologies to compute the relatedness of instances and topics. For this purpose, we need to build a topic ontology that represents the domain of old press articles. In this section, we present Topic-OPA, a topic ontology, harvested from open knowledge graphs, for modeling topics of old press articles.

Generally, knowledge graphs are very large and contain many entities that are too general or specific to be successfully used as topics for topic labeling [6]. Meanwhile, they can be leveraged to build with moderate efforts small to medium-sized meaningful topic ontologies. As a knowledge graph, we selected Wikidata. It is a free and open knowledge graph and acts as central storage for the structured data of its Wikimedia sister projects, including Wikipedia, Wiktionary, and others [13]. Wikidata stores more than 402 million statements about over 45 million entities [29]. Today, more than 60 million items are described. The data model of Wikidata is based on a directed, labeled graph where entities are connected by edges that are labeled by “properties” [5]. Thus, the system distinguishes two main types of entities: items and properties. Items are uniquely identified by a “Q” followed by a number, such as Paris (Q90). Properties describe detailed characteristics of an item and represented by a “P” followed by a number, such as instance of (P31). Entities are represented by URIs (e.g., http://www.wikidata.org/entity/Q90 for Paris and http://www.wikidata.org/entity/P31 for instance of). In the following, we discuss the ontology definition, specification, requirements, and development.

6.The topic labeling process

This section defines the topic labeling process, which is based mainly on RelTopic and Topic-OPA. Given an article Ai∈A represented by a set of non-ambiguous named entities Ni, the topic labeling process of Ai is composed of three main phases: (1) assign Ni as instances of Topic-OPA, (2) apply an instance-topic mapping process, and (3) rank and select the best topics that label Ai.

6.3.Ranking and selection of labeling topics

The ranking and selection of labeling topics is accomplished based on the results of the instance-topic mapping process. For Ai, ∀i∈I, ∃Ti⊂T, ∀t∈Ti, RelTopic(i,t)⩾RelTopic‾(I,T). The matter now is to rank the topics according to these values and select the most relevant topic(s) tk∈Tk⊂Ti for labeling Ai. For this purpose, we define the following procedure:

• 1. Eliminate the non relevant concepts based on three criteria:

  • (a) Level of abstraction: remove most abstract topic concepts such as, Entity, Occurrent and Knowledge, by considering their depths. In Topic-OPA, these concepts’ depths are less than the average of the depths of all the topic concepts.

  • (b) Hypernyms of named entities: remove the topic concepts that are hypernyms of the named entities. For instance, by referring to A1, John Simon is a Politician, thereby concepts such as Professional, Worker, Person, Agent and Individual are eliminated.

  • (c) Hyponyms of general concepts: remove the topic concepts that are hyponyms of Person, Organization, Product and Location. For instance, by referring to A1, Political Activist is related to the instance John Simon. However, Political Activist is not an hypernym of John Simon but a subClassOf Person. Thus, it will be eliminated being an hyponym of Person.

• 2. Compute the most common topic concepts Tc from Tn=∑Ti, ∀i∈I.

• 3. Compute the size of Tc.

• 4. If |Tc|=1, then Tc={tc} is the unique labeling topic of Ai.

• 5. Otherwise, if |Tc|>1 calculate the average of the semantic relatedness values RelTopic‾(i,tc), for RelTopic(i,tc)⩾RelTopic(I,T), ∀tc∈Tc, ∀i∈I.

• 6. Define two strategies to rank Tc and to select the top-ranked topic(s) that label Ai: relatedness-guided and centrality-guided. The relatedness-guided strategy aims to select the most related topic concept(s) according to the relatedness values’ average. Meanwhile, the centrality-guided strategy selects the most connected topic concept(s) based on the degree centrality values. Thus, the further considers the content of Ai, and the latter observes the semantic relevance of the topic concepts. By applying the dual strategy, we extend the selection of the best topics that label Ai.

  • (a) The relatedness-guided strategy is composed of:

    • i. Ranking the topic concepts tc, ∀tc∈Tc according to the average of the relatedness values RelTopic‾(i,tc),

    • ii. Selecting the topic concept(s) tr∈Tr⊆Tc having the highest value.

  • (b) The centrality-guided strategy is composed of:

    • i. Computing the degree centrality of tc, ∀tc∈Tc,

    • ii. Ranking the topic concepts tc, ∀tc∈Tc according to their degree centrality,

    • iii. Selecting the topic concept(s) td∈Td⊆Tc having the highest value.

• 7. Finally, compute the topic labeling set of Ai, Tk=Td∪Tr, as a combination of the results of the centrality-guided and the relatedness-guided strategies.

Fig. 3.

Example of articles from Le Matin.

Fig. 4.

Example of articles from the selected corpus of Le Matin.

Fig. 5.

Example of named entities extracted from {A1,A2,…,A8}.

7.Use-case: Le Matin

In this section, we present a case study for labeling the old French newspaper Le Matin. For this purpose, A={A1,A2,…,A48} a corpus of 48 articles, published between 1910 and 1937, is selected. Every article Ai∈A is described by an XML file consisting of Ni a set of disambiguated named entities represented by Wikidata URIs (see Fig. 5 for an example). Generally, the named entities representing the articles are the outcome of WP2. However, in this work, they are collected manually following the hypothesis presented in Section 2. Besides, T, a set of topics representing all the topic concepts of Topic-OPA, is considered. In Figs 3 and 4, {A1,A2,…,A8} a subset of A is illustrated. Our main goal is to automatically label the articles by applying our proposed semantic relatedness measure RelTopic. In order to achieve the goal, we need to construct the topic ontology Topic-OPA from these articles. Furthermore, the following processes are performed: (1) the assignment of the named entities as instances of Topic-OPA, (2) the instance-topic mapping process, and (3) the ranking and selection process.

7.1.Topic-OPA of Le Matin

For Building Topic-OPA representing Le Matin, a set of N=392 named entities representing A is considered, and the SPARQL-based automatic approach (Section 5.3) is applied. As a result, we obtained a topic ontology, as a subset of Wikidata, which is accessible and manageable in ontology editors such as Protégé.77 Note that the topic ontology is not curated. We maintained the concepts and relations as obtained by the application of the SPARQL-based approach. Thus, Topic-OPA contains 2073 concepts, 3261 SubClassOf relations and 1135 non-hierarchical relations. In Figs 6 and 7, we depict excerpts of Topic-OPA around the Politics and Medicine topics. The solid lines represent the SubClassOf relations, and the dashed lines represent the non-hierarchical relations.

Fig. 6.

Excerpt of Topic-OPA around the concept Politics.

Fig. 7.

Excerpt of Topic-OPA around the concept Medicine.

8.Evaluation and comparison

The first part of this section evaluates Topic-OPA being an application-based ontology. The second part assesses the performance of RelTopic by evaluating the results of the entire framework (Topic-OPA + RelTopic + the topic labeling process). Furthermore, we consider applying the whole approach for labeling recent press articles. Finally, RelTopic is compared to alternative graph-based semantic measures.

8.2.Evaluation of RelTopic

The evaluation of RelTopic consists of measuring how well this measure can label a given corpus of articles. Thereby, we evaluate the performance of the whole framework (Topic-OPA + RelTopic + the topic labeling process) using a dual evaluation approach: (1) a quantitative evaluation that compares the automatic labeling to human labeling [1] and (2) a qualitative evaluation that appraises the generated topics regarding their semantic interpretability [47].

8.2.1.Quantitative evaluation

For evaluating the relevance of the generated topics, a quantitative evaluation is used by considering human-based labeling [1] and rating [24] methods. For this purpose, we considered A as the corpus of 48 articles from Le Matin that we have introduced in Section 7. Since humans can be in contradiction for evaluating specific articles, three different annotators are involved for labeling and rating each article, Ai∈A. Concerning the labeling process, the textual content of the articles Ai∈A is assigned to the human annotators. The humans who were blind to Topic-OPA and the results generated by RelTopic, have read the articles and assigned (multi-) labeling topics based on the content (see Table 4). Based on human labeling, an inter-annotator evaluation is established to compute the agreement among the annotators for each Ai∈A. A comparison of the assigned topics (performed in the context of Wikidata) has shown an agreement of 46% for exact topics (e.g., A3, A5, A6), 26% for specific/general (e.g., A7, A8), and 15.5% for semantically related (e.g., A1, A2, A4). Furthermore, we compared RelTopic topics (see Table 3) with those assigned by humans. Our approach manifested an agreement, with human labeling, of 42% for exact topics (e.g., A5, A6), 34% for specific/general (e.g., A3, A7, A8), and 6% for semantically related (e.g., A1, A4).

Table 4

Excerpt of human labeling of the articles represented in Table 3

Ai	Annotator1	Annotator2	Annotator3
A1	International Politics	International Relations	International Politics
A2	Politics	Foreign Policy	Politics
A3	Cinema	Art, Cinema	Cinema
A4	Higher Education	Politics, Education	Politics, Science
A5	Aviation	Event, Exploration, Aviation	Aviation
A6	Cycling	Sport, Cycling	Cycling
A7	Medicine	Science, Medicine	Science, Vaccination
A8	Economics	Economics	Finance

To resume, our approach has an agreement quite close to the annotators’ agreement. For the rating method, the humans are asked to rate RelTopic labels for each Ai∈A using the following scores [24]: 3 for very good labels; 2 for reasonable labels; 1 for semantically related labels, but not considered as good topics; 0 for inappropriate labels. As a result (see Table 5 for an example), 36% of the RelTopic topics are assessed as very good, 40% as reasonable, 14% as semantically related, and 10% as inappropriate.

Table 5

Excerpt of human rating of the articles represented in Table 3

Ai	Annotator1	Annotator2	Annotator3
A1	2	2	2
A2	0	0	2
A3	2	3	2
A4	3	2	2
A5	3	2	3
A6	3	3	3
A7	2	2	3
A8	3	3	2

In the following, we analyze the cases where RelTopic produced general or irrelevant labels considering the validity of the named entities. In this context, two main issues are observed: (1) the existence of not disambiguated named entities; (2) the lack of some types of named entities. For this purpose, two additional articles are considered A9 and A10 (Fig. 8).

Fig. 8.

Le Matin.

The existence of not disambiguated named entities In the presented use-case (Section 7), 20 articles have been represented by some named entities that are not disambiguated (e.g., A5, A7). In this section, we discuss the influence of these named entities on the relevance of the automatically generated labeling topics. First, we analyzed two articles A7 (Fig. 4) and A9 (Fig. 8a). A7 consists of 5 disambiguated named entities and 2 that are not disambiguated. Despite this default, RelTopic assigned Vaccination (see Table 3) as a specific topic compared to Medicine assigned by humans. Second, we considered article A9 which consists of 10 disambiguated named entities and 2 that are not disambiguated. By applying RelTopic, A9 is labeled by Science (step 4 of the topic labeling process). However, Medicine is assigned by the human annotators. By surveying the results of the instance-topic mapping phase and the computation of the common related topics, we found that Medicine is commonly related 8 times. Meanwhile, Science is commonly related 10 times. In addition, we have inspected the named entities that are not disambiguated in A9 (Robert Wilbert and Marcel Léger). Robert Wilbert is a Veterinarian88 and Marcel Léger is a Epidemiologist, Microbiologist and Bacteriologist.99 We conclude that the existence of these not disambiguated named entities has eliminate Medicine from the most common topics set. Thereby, they have affected the labeling relevance degree of A9.

The influence of the lack of named entities types As evoked earlier, in this study, we are interested in three main types of named entities: person, organization and product. In this section, we discuss the influence of the lack of some types on the relevance of generated topics. For instance, article A10 (Fig. 8b) is composed of 6 persons and 2 products and the majority of persons are politicians (see Table 6). Thereby, A10 is labeled by Politics (step 4 of the topic labeling process). However, based on the content and the subject of A10, the human annotators have assigned the topic Economics. In this context, we recognized that most politicians, with the absence of organizations or persons related to economics, have affected the labeling results’ pertinence.

Table 6

Assignment of the disambiguated named entities of A10 as instances of Topic-OPA

Article	Named Entity	Instance of
A10	César Caire	Jurist⊓Lawyer
	Henri Galli	Politician⊓Journalist
	Emile Desvaux	−
	Ambroise Rendu	Politician
	Alexandre Luquet	Politician
	Flour	FoodIngredient
	Wheat	FoodIngredient

8.2.2.Qualitative evaluation

The qualitative evaluation assesses the labeling topics generated by RelTopic according to their semantic quality [47]. In linguistics, the topic, or theme, of a sentence is what is being talked about.1010 In a semantic context, defining a labeling topic within topic ontologies is not an easy task. In fact, a topic ontology consists of various concepts including the labeling topics. Meanwhile, it is difficult to find or define these topics. In our experiment, by the application of RelTopic for labeling the old press articles (see Table 3), we perceived three essential characteristics that define the semantic quality of a labeling topic:

• – Highly correlated: a concept with high degree centrality designates a large surface of connection with the concepts within the ontology. For instance, Politics, War, Science, Art and Sport have respectively 29.17, 22.22, 23.62, 31.34 and 13.89 values of degree centrality. Meanwhile, concepts such as Activity, Occupation and Group Behaviour have respectively 8.68, 9.81 and 7.63 values of degree centrality.

• – Core concept: the depth of concepts in ontologies indicates their degree of generality. In Topic-OPA, abstract concepts, such as Entity, Agent, Object, Product and Occurrence are located at depths less than the average of depths in Topic-OPA which is equal to 4 (e.g., depth(Entity)=1, depth(Object)=2 and depth(Occurrence)=3). These concepts are not recommended as labeling topics due to their abstraction interpretability. Meanwhile, the majority of the labeling topics that are produced by our relatedness measure (e.g., Politics, Art, Science, etc.) are located at depths greater than or equal to the average of depths in Topic-OPA (e.g., depth(Politics)=5, depth(Art)=4 and depth(Science)=5). Although, these topics are more general than the specific concepts (e.g., Contract Law, Pharmacy, etc.) which are located at higher levels of depth (e.g., depth(Contract Law)=7 and depth(Pharmacy)=9).

• – Not a hypernym of named entities: a labeling topic is not linked hierarchically to the named entities. Therefore, it is not a subclass of Person, Organization, Location or Product.

8.4.Comparison of RelTopic with alternative graph-based measures

In this section, we compare RelTopic (Equation (8)) with alternative graph-based measures. Specifically, we choose path-based measures since node-based measures are dependent on textual resources, which are out of the scope of our study. To analyze the importance of semantic relatedness regarding semantic similarity, we compared RelTopic to SimRada (Equation (1)). Thus, SimRada is applied to the whole graph of Topic-OPA, including the hierarchical and non-hierarchical schemes. Besides, a comparison with the most commonly known semantic relatedness measure RelHS (Equation (3)) is addressed. For applying RelHS, there is a need to compute each link’s direction change (hierarchical and non-hierarchical) through all the paths. However, this computation is considered a difficult task [30]. To simplify, we computed the direction changes of the hierarchical edges only. Furthermore, we compared the results of applying these measures regarding the instance-topic mapping process on A. Table 7 shows an excerpt of the results of mapping A7 to Topic-OPA. The results imply that ti the topic concepts related to i, ∀i∈Ii (Ii are the instances associated to Ai∈A) are identified by RelTopic as well as by SimRada and RelHS (e.g., Education and Research are related to i=Académie Nationale de Médecine in A7, Fig. 9). However, the use of RelTopic and RelHS makes also evident the identification of the topics that are not related to i, ∀i∈Ii due to the considerable gap among the relatedness values (e.g., Economics and Business are not related to i=Académie Nationale de Médecine in A7). Besides, the results obtained by RelTopic and RelHS are close. Nevertheless, the computation of semantic relatedness using RelTopic is undemanding regarding the edges’ direction changes. For an accurate comparison, the relatedness values of RelHS (∈[0,8], Equation (3)) are converted to [0,1] (division by 8).

Table 7

Excerpt of the results of the instance-topic mapping process of A7 to T

Instance (i)	Topic Concepts (ti)	SimRada	RelHS	RelHS/8	RelTopic(i,ti)
Académie Nationale de Médecine	Research Institute	0.5	7	0.87	0.81
	Education	0.33	6	0.75	0.7
	Research	0.33	6	0.75	0.76
	Higher Education	0.25	5	0.62	0.57
	Business	0.14	3	0.37	0.32
	Economics	0.16	3	0.37	0.33
Albert Calmette	Physician	0.5	7	0.87	0.73
	Medicine	0.33	6	0.75	0.66
	Immunology	0.33	6	0.75	0.64
	Science	0.25	4	0.5	0.59
	Business	0.16	3	0.37	0.31
Alfred Boquet	Physician	0.5	7	0.87	0.74
	Veterinary Medicine	0.33	6	0.75	0.69
	Science	0.25	4	0.5	0.61
	Business	0.16	3	0.37	0.31
BCG vaccine	Vaccination	0.33	6	0.75	0.71
	Medication	0.33	6	0.75	0.68
	Health Care	0.25	4	0.5	0.48
	Business	0.14	2	0.25	0.3

Fig. 9.

Comparison of the results of the instance-topic mapping of A7 (Académie Nationale de Médecine).

9.Discussion

This study’s main contribution is the design of a novel graph-based semantic relatedness measure, named RelTopic, for topic labeling purposes. By proposing RelTopic as a hybrid measure, we contributed to overcoming node-based and edge-based approaches’ limitations. RelTopic considers hierarchical and non-hierarchical relations and inspects the semantic properties of entities within topic ontologies. Thus, we considered the correlation of nodes to overcome the dependency of measures to textual resources. Besides, we separated hierarchical and non-hierarchical edges using different weights to overcome the limitation of equality of edges in path-based approaches. RelTopic takes as inputs two entities (e.g., instances and concepts) and returns a numerical value representing their relatedness according to a topic ontology. In this work, RelTopic is applied mainly for labeling old press articles by assessing the relatedness of instances (named entities) and topic concepts in the topic ontology. Besides RelTopic is reused for labeling different articles, recent newspapers. The reusability of RelTopic for purposes requiring the computation of semantic relatedness between entities in a given ontology is demonstrated. However, for this purpose, two main factors are mandatory: (1) a reasonable characterization of the application domain using a domain ontology and (2) a definition of the input entities that should be included in the ontology (e.g., concept-concept, instance-concept).

This study’s second contribution is developing the general topic ontology Topic-OPA using a SPARQL-based automatic approach. Topic-OPA is harvested from open knowledge graphs (e.g., Wikidata) based on a set of disambiguated named entities representing the application domain. Topic-OPA is a domain-dependent topic ontology since it is developed from the named entities of the given domain. Nevertheless, if Topic-OPA is developed from all the named entities of the application domain (e.g., Le Matin), it could be reused as a topic ontology for labeling old press articles of any journal or newspaper belonging to the same period of time. We assume that approximately the same types of persons (e.g., politician, diplomat, actor, physician, botanist, etc.), organizations (e.g., bank, public treasury, academy, etc,), or products (e.g., vaccine, film, etc.) are available during a comparable period of time (e.g., 1910–1945). Besides, the SPARQL-based approach is reusable (as shown in Section 8.3 in the case of recent newspapers) for harvesting ontologies from open knowledge graphs, requiring the starting named entities representing the domain of discourse.

Finally, a significant contribution is applying an ontology-based automatic topic labeling approach for labeling press articles. This process, which is composed of a topic ontology and the semantic relatedness measure RelTopic, is generalizable for implementing labeling activities for any text, including newspaper or magazine articles. As demonstrated in this work, the entire approach is applied for labeling articles in two different contexts, old and recent press. A primary requirement for the approach reuse is the availability of the named entities representing the text to be labeled. These entities, which are independent of any language, will permit a topic ontology building representing the domain. Thus, RelTopic will assess the relatedness of each text’s named entities to the topics of the topic ontology. Finally, a selection process of the best topics is performed to label the textual resources.

In this context, an important question arises. What if there is a lack of named entities, or if they are ambiguous or inexact? This situation contemplates the validity of this work’s general hypothesis (see Section 2). In Section 8.2.1, we analyzed the influence of two issues on the generated results: (1) existence of not disambiguated named entities and (2) lack of some types of named entities. Both of them had an impact on the generated topics. However, this impact is relative depending on the named entities representing the text to label. For example, the first issue has affected the generality of the assigned topic (e.g., Science is given instead of Medicine). Meanwhile, the second issue has affected the relevance of the assigned topic (e.g., Politics is given instead of Economics).

To resume, the relevance of the whole framework’s outcome is a crucial measure of the validity of this work’s hypothesis. Thus, the given named entities representing the articles are valid if RelTopic and the whole framework achieves relevant labeling topics. This assumption is demonstrated in two different contexts, old and recent press.

10.Conclusion

The task of automatically labeling newspaper articles according to a predefined set of topics is a challenging research issue, specifically in cultural heritage. A pertinent characterization of the application domain is required for this purpose. In the context of the ASTURIAS project, which aims to label a vast number of old press articles automatically, we envisaged graph-based semantic measures. These measures have shown effective results in different areas such as knowledge engineering, Semantic Web, and Natural Language Processing. Graph-based semantic measures are composed of similarity and relatedness measures. The former class is adapted to taxonomies and widely investigated in the community. The latter class is adapted to ontologies, and few attempts have been found in the literature to design such measures. Designing semantic relatedness measures is a challenging research task. Nevertheless, they are valuable since they inspect the semantic properties of entities in ontologies.

In this study, we proposed a novel semantic relatedness measure, named RelTopic, within topic ontologies for topic labeling of old press articles. In contrast to existing measures, RelTopic considers hierarchical and non-hierarchical relations and assesses the relatedness between instances and concepts. To apply RelTopic, we considered topic ontologies as weighted graphs where nodes and edges are given positive numerical weights. Besides, RelTopic considers the degree centrality of nodes, which reflects the node’s surface of connection with regards to the rest of the ontology. For the application of RelTopic, a topic ontology, named Topic-OPA, representing the domain of old press articles, is harvested from Wikidata by applying a SPARQL-based automatic approach.

The proposed approach is evaluated using a dual evaluation approach. First, a quantitative evaluation is performed with the help of three different annotators. The human annotators have assigned labels to a corpus of 48 articles from Le Matin. Our approach has shown an agreement quite close to that shown by humans for exact, specific, or general topics. Furthermore, the annotators have rated the results of RelTopic regarding their relevance. We obtained 76% of the generated topics are rated as very good and reasonable. The second phase of the evaluation consists in applying a qualitative approach that appraised the semantic interpretability of the automatically generated topics. We noticed that the topic labels within Topic-OPA are highly correlated and located at the ontology’s core level. Additionally, the reuse of the entire approach is demonstrated for labeling recent newspaper articles. Promising results are achieved endorsing the reusability of the labeling approach using RelTopic in different domains. Finally, we compared RelTopic to alternative graph-based semantic measures. The strength of RelTopic is its capability to clearly identify the related topics from the non-related topics with an undemanding computation of direction changes of paths.

In future works, we will be interested in the following tasks. First, the contextualization of the articles is envisaged taking into account the named entities of type location (e.g., A1 could be labeled with International Politics, A3 with Local or French Art and A6 with French Sport). In this study, we do not consider the topic ontology’s curation; we maintained the ontology structure and content, including the abstract and specific concepts, as derived from Wikidata. In further work, we will apply a curation process to clean and leverage Topic-OPA. Furthermore, we will study the application of RelTopic on the leveraged version of Topic-OPA and assess the generated labeling topics’ quality.