A survey on knowledge-aware news recommender systems

3.1.1.Search queries

We defined two queries, targeting the task of (Q1) news recommendation and the usage of (Q2) external knowledge, in order to collect relevant literature. Table 1 illustrates the search strings used for each of the two queries. Keywords meant to capture (Q2) external knowledge include multiple terms referring to widely used sources of knowledge, such as knowledge graphs or ontologies. As such, the results of query (Q2) are given by the union of the results of the corresponding search strings. Since we are interested only in news recommender systems that use a form of external knowledge, the final query used in the publications’ search process represents the intersection of queries (Q1) and (Q2).

3.1.2.Sources

The following bibliographic databases and archives constitute the sources used for the literature search: (i) DBLP11 (ii) ACM Digital Library22 (iii) IEEE Xplore33 (iv) Science Direct44 (v) Springer Link55 (vi) Web of Science66

3.1.3.De-duplication

The results collected from the previously specified sources are then merged and de-duplicated in a threefold process. Firstly, for all the publications retrieved during the keyword-based search, we gather the associated bibtex files produced by each of the digital libraries and store them using the Zotero77 bibliographic tool, which also performs automatic detection of duplicates based on the papers’ metadata. Secondly, we serialize as a spreadsheet the retrieved publications and their metadata, including title, DOI, abstract, authors, publication venue, and date. Lastly, we use the spreadsheet to perform manual de-duplication of the papers which could not be detected by the bibliographic tool due to large differences in their metadata, such as the publication venue which can be reported by some digital libraries as the conference venue and referred to by others as Lecture Notes in Computer Science (LNCS). The manually de-duplicated results of the keyword-based search constitute the final list of papers used for selecting the relevant literature in the next step.

3.2.1.Selection criteria

The list of inclusion criteria displayed in Table 2 was developed based on the goals of the survey in order to filter out irrelevant publications. Each criterion is composed of both an inclusion criterion (IC) and an exclusion criterion (EC). A paper needs to fulfill all inclusion criteria to be selected for the study.

3.2.2.Selection process

The study selection process is composed of two phases.88 Firstly, relevant papers are pre-filtered based on their metadata. More specifically, the validity of criteria C1-C4 is assessed by examining the publications’ language, publishing year, venue, type, keywords, title and abstract. The validity of the remaining criteria is also checked if the information contained in the metadata allows it. Papers not fulfilling all of these criteria are excluded from the rest of the study. Papers whose relevance cannot be determined solely from the metadata will be kept until the next stage of the selection process. In the second phase, the fulfillment of criteria C5-C9 is checked using the complete content of the pre-filtered publications. The papers meeting all the requirements of this phase will be included in the final set of publications for the survey. Table 3 shows the number of papers remaining after different stages of the selection process.

5.2.1.Types of recommendation techniques

News recommendation systems generally adopt one of the three main techniques for predicting whether a user will interact with a certain article, namely content-based, collaborative filtering, and hybrid. However, content-based approaches are the most widely used in the field of news recommendation [84].

5.2.2.Knowledge base

The knowledge resources used by knowledge-aware recommender systems can be grouped into domain ontologies and knowledge graphs. In the remainder of the paper, these will be referred to as knowledge bases (KB), if the type of resource is not explicitly specified. The former category can be further split into self-constructed ontologies – built either from combining smaller domain ontologies or subsets of large knowledge bases (e.g. DBpedia [92], Hudong encyclopedia [177]) or directly from news articles (i.e. financial domain ontology using information from Yahoo! Finance [78]) – and controlled vocabularies used in the news domain, such as the IPTC News Codes99 [160].

In the latter category, one can distinguish between open source and commercial knowledge graphs. In the first subgroup, cross-domain knowledge graphs such as Wikidata, DBpedia, and Freebase are widely used in news recommender systems. Freebase [12] was initially launched by Metaweb in 2007, and later acquired by Google in 2010, before being shut down in 2015 [126]. The latest version of Freebase, available at Google’s Data Dumps1010 has been estimated to contain 1.9 billion triples [63]. Wikidata [167], a collaboratively edited knowledge graph, containing several language editions of Wikipedia, as well as data previously contained in Freebase [126], consists of 92 million items1111 and over 1174 million statements.1212 DBpedia [92] is a knowledge graph built by extracting structured data from various language versions of Wikipedia, and contains in its most recent and largest version, DBpedia Largest Diamond, 220 million entities and 1.45 billion triples.1313

WordNet [111], a large English lexicon containing nouns, verbs, adjectives, and adverbs grouped into synsets (i.e. sets of synonyms), which are further interconnected via semantic relations of antonymy, hyponymy, meronymy, troponomy, or entailment, is often used in knowledge-aware news recommender systems for word sense disambiguation. More specifically, each word in WordNet is associated with a set of senses, which denote the set of possible meanings that the word might have. For example, the noun “Jupiter” can refer to either the planet in the solar system or the supreme god of the Romans. WordNet 3.01414 contains 117,659 synsets and 206,941 word-sense pairs.

In the subgroup of commercial knowledge bases, Satori [10], the knowledge graph proposed by Microsoft, is the most often used one, especially by recent deep learning-based news recommender systems. Although very little information about the data contained in Satori is publicly available, it was estimated to contain in 2012 approximately 300 million entities and 800 million relations [126].

5.2.3.Structure of knowledge base

News recommendation models use knowledge bases by exploiting their different structures in order to extract either semantic, structural, or both types of information. A few knowledge-aware news recommender systems exploit only the semantic information contained in a knowledge graph or ontology, by extracting concepts or entities that appear in a news article, which will be denoted as concepts/entities only models for the rest of this article. A larger share of models however enriches the basic set of knowledge entities by expanding it with the neighborhoods of extracted entities in the knowledge graph and by considering the paths and relationships between entities (denoted as entities + paths). Another method for enhancing the set of concepts or entities extracted from a knowledge base is by taking into account its structure, namely the different types of relations between nodes in an ontology, such as synonymy or hyponymy relationships in semantic lexicons, or the distances between concepts, entities or classes (denoted as concepts + KB structure or entities + KB structure). Differently from these categories of models, the newer deep-learning-based recommendation techniques exploit simultaneously both the semantic and the structural information encoded in knowledge graphs, by means of knowledge graph embeddings (denoted as entities + KG structure).

5.2.4.Target function

Two main target functions can be distinguished in news recommendation models, namely click-through rate (CTR) prediction and item ranking. Models classified in the first group aim to predict the probability that the user will click on the target article, whereas methods in the second group recommend the top N most similar articles to the articles previously read by the user.

5.2.5.Addressed challenge

In addition to enhancing the accuracy of recommendations, knowledge-aware news recommender systems aim to address different challenges of the news domain or limitations posed by conventional recommendation techniques. Several news articles, written in different manners, using semantically related terms, can describe the same piece of news, and numerous words have different meanings depending on the context in which they are used. While humans can easily distinguish ambiguous words, or words connected via certain semantic relations, such as synonyms, this constitutes a challenge for recommendation models using text representations. Knowledge-aware recommender systems propose to remove such ambiguity from text by representing an article using only disambiguated knowledge entities or concepts from a controlled vocabulary, instead of all the terms. In turn, this leads to faster computations, since the model is required to consider a limited number of concepts or entities, which is significantly smaller than the total number of words contained in an article. Moreover, the semantic meaning of news, as well as the semantic relatedness of concepts (i.e. news describing similar or related concepts might indicate different interests of a user) can be captured by further considering the relations between the different concepts found in an article.

News articles contain a large number of named entities, used to denote information regarding the events described, such as the location, actors involved, time, or what the event refers to. However, named entities are not taken into account in traditional text-based recommendation models. In contrast, knowledge-aware techniques handle named entities by extracting them from the text and enriching them with external information encoded in knowledge graphs. Furthermore, using external information for recommendation can help overcome the data sparsity and cold-start problems, as articles can be connected using relations in the knowledge graph between the entities extracted from text, such that new items without user feedback can also be included in the recommendations.

Moreover, injecting external knowledge into the recommendation model has three additional benefits. Firstly, it extends text-level information with common sense knowledge which is encoded in knowledge graphs but cannot be extracted only from an article’s text. For example, a user reading the titles of the news articles in Fig. 1 will probably know that Elon Musk and Robinhood were participants in the GameStop short squeeze event that affected GameStop, or that the New York Stock Exchange is located on Wall Street. However, a text-based recommendation model does not possess such common knowledge information. Additionally, using external information also helps the model discover latent knowledge-level connections between the news, such as the fact that the two snippets in the example from Fig. 1 are connected, although they do not appear related when considering only the words in their titles. Lastly, exploiting the knowledge-level and semantic connections between news can improve the diversity of recommendations, as the model learns to avoid recommending articles that are too semantically similar, even if they are published in different sources and have different writing styles.

Fig. 1.

Illustration of a knowledge graph-enhanced news recommender system (reproduced from [170]).

6.1.1.Overall framework

Non-neural, entity-centric methods first create a vector representation of both the target article and the user profile, where the latter consists of the user’s reading history. Afterwards, the models compute the similarity between the two representations and recommend a list of the top N articles whose similarity scores exceed a predefined threshold. As such, the majority of techniques listed here adopt a content-based recommendation approach. We analyze these systems in terms of three differentiating factors:

6.1.2.Representative models

In this subsection, we discuss 12 representative non-neural, entity-centric recommendation techniques.

Cantandor et al. [21,23] developed a semantic context-aware recommendation model which aims to contextualize the users’ interests, such that the model learns to ignore preferences that are out of focus in a particular session, and to place a higher importance on those that are in the semantic scope of the ongoing user activity. The profiles of both user and news articles are described using semantic concepts from a domain ontology, as u=(w1u,w2u,…,wqu), and v=(w1n,w2n,…,wpn), respectively. The concepts in the user’s vector representation are weighted with weights wiu∈[−1,1], which indicate the intensity of the user’s interest for concept ci∈O. A negative weight is equivalent to a dislike for the concept, while a positive value shows that the user is interested in the given concept. Similarly, concepts weights win∈[0,1] place the article’s representation in the same vector space as the user’s preferences [23].

A personalized content retrieval approach assigns a relevance measure pref(v,u)=cos(v,u) of an item v to user u using the cosine similarity between their vector representations. However, a good recommendation model should be able to differentiate between a user’s short and long-term preferences, which could be accomplished by enhancing it with contextualized semantic preferences. More specifically, Cantandor et al. [23] define a semantic runtime context as the background topics Cut under which user u performs a set of activities in the unit of time t. The runtime context, illustrated in Eq. (3), comprises a set of weighted concepts from a domain ontology, collected from the articles accessed by the user during a session.

Following the construction of the runtime context, Cantandor et al. [23] introduce a semantic preference spreading strategy which expands the user’s initial preferences through semantic paths towards other concepts in the ontology. This contextual activation of user preferences constitutes an approximation of conditional probabilities. According to this formulation, the probability that concept ci∈O is of interest for a user is determined by the probability that the concept ci itself, as well as all other concepts cj∈O directly connected to it in the ontology, belong to the same topic, and the probability that the related concept cj is also relevant for the user.

Consequently, the semantic spreading mechanism requires weighting every semantic relation r in the ontology with a value w(r,ci,cj) which denotes the probability that concepts ci and cj belong to the same topic given the fact that they are connected by relation r. The initial set of user preferences expressed in terms of concepts, Pu={ciu∈O|wku≠0}, is expanded as follows:

The context-aware personalized recommendation model computes the relevance measure of an item v for user u using the expanded profiles of the user and the article, in the following way: prefc(v,u)=λ·pref(v,EPu)+(1−λ)·pref(v,ECu). In this case, the parameter λ∈[0,1] weights the strength of the personalization component with regards to the current context.

The weights spreading strategy addresses both the cold-start and the data sparsity problems, whereas incorporating contextual information captures the changing utility of a news article to a user based on temporary circumstances. While this model applies to single users, Cantandor et al. [23] also employ a hybrid context-aware recommendation technique which exploits the connections between users and concepts to discover relations among users in a collaborative fashion. The goal, in this case, is to leverage partial similarities between users with similar preferences in a focused domain, but who are globally dissimilar. On a high level, the strategy is accomplished by clustering users according to layers of preferences shared among them. Hence, the user similarities depend on sub-profiles, which increase the likelihood of extracting conjunctions of rare preferences.

Semantic Communities of Interest are derived from the users’ relations at different semantic levels [23]. More specifically, each ontology concept ciu occurring in a user’s reading history is represented as a vector of weights measuring its importance for the user, namely ciu=(w1,i,w2,i,…,wM,i)∈[−1,1]M. A hierarchical clustering method is used to determine groups of preferences in the concept-user vector space, and each user is assigned to a concept cluster based on the similarity of his profile to cluster Cq, computed as sim(u,Cq)=∑ciu∈Cqw1u|Cq|, where ci is the concept associated with the wiu element in the user’s preference vector.

Cantandor et al. [23] propose two recommendation models that use the extracted latent communities of interests among users. On the one hand, model UP computes a unique ranked list of news articles based on the similarities between news and all semantic clusters, meaning that it compares a user’s interests to those of the other users and utilizes these user-user similarities to weight preferences for candidate articles. As such, the preference score of article v to user u is computed using Eq. (5):

Here sim(v,Cq)=∑ci∈Cqwinv|Cq| represents the similarity between item v and cluster Cq, and nsim(v,Cq) denotes the normalized similarity over the set of all clusters. Moreover simq(u,y) and nsimq(u,y) are the single and normalized similarities at layer q between users u and y, defined as the cosine similarity of the projections of their corresponding concept vectors onto cluster Cq. Therefore, model UP takes into account both the characteristics of news articles, as well as the relations between user, at different semantic layers.

On the other hand, model UP-q generates recommendations separately for each layer by computing a ranked list for each semantic cluster. The preference between user and target article is calculated as follows:

The same context-aware and multi-facet, group-oriented hybrid recommendations are also adopted by Cantandor et al. [24] to generate social tags enriched recommendations. The authors expand the original user profiles with personal tag clouds collected from two websites (Flickr and del.icio.us). The extracted tags are incorporated into the ontological user profiles by mapping them to ontology concepts.

Concept Frequency – Inverse Document Frequency (CF-IDF) [64] constitutes a variant of the TF-IDF weighting scheme that uses concepts instead of terms in order to represent news articles. In the framework proposed by Goossen et al. [64], the profile of a user u consists of a set of q concepts from an ontology, namely u={c1u,c2u,…,cqu}, ciu∈O. In turn, each concept cu from the user profile is represented as a set of k news articles vj in which it occurs, namely cu={v1,v2,…,vk}. An article is thus composed of a set of p concepts occurring in it, denoted as v={c1n,c2n,…,cpn}, cn∈O.

In the CF-IDF recommender, each user’s interests are represented as a vector of CF-IDF weights wu corresponding to the concepts appearing in the user’s previously read articles, as shown in Eq. (7). Analogously, the article’s profile is computed according to Eq. (8).

The CF-IDF weights are computed similarly to TF-IDF weights. Firstly, the Concept Frequency cfi,j calculates the frequency of a concept ci in an article vj as the ratio between the number of occurrences in the given article, ni,j, and number of occurrences of all concepts appearing in the article, nk,j. Since highly frequent concepts are less informative than rarer ones, the Inverse Document Frequency idfi penalizes such concepts by increasing the weight of concepts rarely occurring across all |D| articles in the corpus. For a concept ci, this is achieved by computing the logarithmically scaled inverse fraction of documents containing the concept. The final weight is given by multiplying the two components according to Eq. (9).

A major difference between the TF-IDF and CF-IDF lies in the fact that the latter considers only the ontology concepts contained in the text, instead of all the terms. Therefore, it assigns a larger value to the concepts deemed more important, and results in faster computations, as it considers a smaller amount of elements during similarity computations. In turn, this also implies that CF-IDF can assign a single representation to multi-word expressions (e.g. “Elon Musk” is one concept), compared to TF-IDF which would compute an embedding for each word individually (e.g. for “Elon” and for “Musk”). Consequently, if the concepts are disambiguated, CF-IDF can handle ambiguous terms, in contrast to TF-IDF.

The Synset Frequency – Inverse Document Frequency (SF-IDF) approach of Capelle et al. [26] modifies the TF-IDF weighting scheme to take into account the semantic meaning of terms in a text. In comparison to CF-IDF, Capelle et al. [26] represent the user’s and article’s profile as sets of WordNet synsets of the terms appearing in the news article. Mathematically, the news item’s profile is represented as:

The synsets in both profiles are weighted using SF-IDF weights, obtained from TF-IDF by replacing terms with synsets s, i.e. sf-idfs,d=sfs,d×idfs,d. The main advantage of SF-IDF is that the same synset is used to represent words with identical meaning, thus reducing the ambiguity of terms and taking into account their semantic relatedness.

However, SF-IDF yields a limited understanding of the semantics of news. Therefore, SF-IDF+ [112] additionally considers the semantic relationships between synsets in order to overcome this drawback. This is achieved by extending a set of synsets S(s) with the concepts connected through semantic relationships with the included synsets, as shown in Eq. (12), where s denotes a synset, R(s) represents the set of relationships of this synset extracted from a semantic lexicon, such as WordNet, and r(s) indicates the corresponding synset according to relationship r.

Hence, the item’s and user’s profiles, v and u, are extended according to Eqs (13) and (14), respectively.

Furthermore, SF-IDF+ not only uses extended synsets instead of synsets, as is the case for the SF-IDF model, but also assigns different weights wr to the relationship r connecting a synset with its semantically related synset, as per Eq. (15).

A similar strategy to the one proposed in SF-IDF+ is also adopted by CF-IDF+ [40], an extension of the original CF-IDF model in which the initial set of identified concepts is expanded with related neighbors. Nonetheless, a shortcoming of the SF-IDF+ recommendation model is not being able to take into consideration named entities, which are prevalent in news articles. Thus, Capelle et al. [28] proposed Bing-SF-IDF+, a method which extends the SF-IDF+ technique with named entity similarities computed using Bing page counts. The main assumption made by the authors is that the likelihood of two entities being similar is directly proportional to the amount of times they co-occur on websites [28]. The Bing-SF-IDF+ similarity score combines two elements, namely the Bing component which measures the similarity between pairs of named entities,1515 and the SF-IDF+ component which computes the similarity between synsets.

The SF-IDF+ profiles and weights are built and calculated according to Eqs (12)-(15). For the Bing component, new user and item profiles are built using sets of named entities extracted from the text with a named entity recognizer, denoted as follows:

Subsequently, the Bing search engine is used to compute the page count c(en,eu) for each pair (en,eu) of named entities in V, namely how many pages found by querying Bing contain either one or both of the entities in a pair. A page rank-based Point-Wise Mutual Information (PMI) [16] co-occurrence similarity measure is afterwards used to calculate the difference between the actual and the expected joint probability of the occurrence of a pair of named entities in a query on a web search engine [28]. PMI assumes independence between the two named entities and is based on their marginal probabilities, as illustrated in Eq. (19).

The SF-IDF+ similarity simsf-idf+(u,v), and the Bing similarity simBing(u,v) scores, namely the cosine similarity of the user and target news article profiles, are then normalized using a min-max normalization between 0 and 1 in order to ensure compatibility of scores. Lastly, the Bing-SF-IDF+ score is defined as the weighted average of the two components’ normalized similarity scores, according to Eq. (21).

This approach of enhancing semantics-driven recommender systems with named entity similarities using the Bing page counts are prototypical also for other models, such as Bing-SF-IDF [71], Bing-CF-IDF+ [17], or Bing-CSF-IDF+ [161].

An approach combining CF-IDF and SF-IDF, which aims to address the ambiguity problem by representing news articles using key concepts, synonyms, and synsets from a domain ontology, is represented by the OF-IDF method proposed by Ren et al. [133]. In this case, a news article is described in terms of key concepts contained in a financial domain ontology. Additionally, the lexical representation of a concept is disambiguated by enriching it with its corresponding synset retrieved from WordNet. Similar to CF-IDF, the concepts in the article’s profiles are weighted using an Ontology Frequency-Inverse Domain Frequency scheme. Thus, the article can be represented as a vector of OF-IDF weights wn associated with the concepts it contains, namely v=(w1n,w2n,…,wpn), where the weights are computed as follows:

In Eq. (22), ni,j is the number of occurrences of concept ci in article j, and 1⩽i⩽p, where p denotes the total number of concepts in j. The user’s interests in the read news can be described by means of a user-concept matrix whose rows denote the read articles, columns indicate the concepts appearing in these articles, and the entries correspond to OF-IDF weights. According to Ren et al. [133], such a user-concept matrix can be modified using relevance feedback in order to capture different interaction patterns between a user and a target article. More specifically, the original OF-IDF weights are adjusted depending on whether the user clicked, read and liked, or read and did not like an article. Under this assumption, the user profile is changed as follows:

In Eq. (23), the vectors Sα, Sβ, Sγ represent the m articles clicked, n articles read and liked, and respectively, l news read and not liked by the user. These vectors of OF-IDF weights are modified using parameters α, β, γ, where the first two parameters are positive to illustrate the user’s interest in an article, while the last one is negative to capture negative feedback.

The semantic relatedness model of Getahun et al. [60] compares two articles vi and vj using the cosine similarity of their vector representations vl comprising of concepts from an ontology and their corresponding weights:

In comparison to item profiles of models such as CF-IDF (Eq. (8)), in Eq. (24) the total number of concepts appearing in an article’s profile is represented by the number of distinct concepts p=|CSi∪CSj| in the sets denoting the two texts, CSi and CSj, respectively. The weight wi of concept ci is based on its occurrence in the set of concepts CSj of the other text. More specifically, if ci is contained in CSj, then it receives a weight of wi=1, otherwise its weight is determined by its maximum enclosure similarity to concept cj. Mathematically, this condition is expressed as follows:

The enclosure similarity between two concepts ci and cj represents the fraction of concepts from the semantic neighborhood of concept ci that also appear in the semantic neighborhood of concept cj. The advantage of this method is that it takes into account related concepts of a concept appearing in news, by utilizing its global semantic neighborhood.

The Ranked Semantic Recommendation (RSR) [78] model is based on the assumption that reading an article containing a certain concept expands the user’s knowledge not only in that particular concept, but also in the concepts related to it. This notion is captured by assigning a rank to each concept from an ontology. For example, a user reading news about a concept represented by the class instance Robinhood might also be interested in his CEO Vladimir Tenev or in the GameStop short squeeze event. Since these instances are in a direct relation to Robinhood, the ranks of all three should be increased. Similarly, if a user firstly reads an article containing instances Robinhood and Elon Musk, then accesses news about Open AI, a related concept instance to Elon Musk, but not to Robinhood, the rank of Elon Musk should be increased, while that of Robinhood should be decreased. Therefore, the rank of a concept aims to account for the user’s changing interests.

Each concept ci is associated with a set of related concepts r(ci)={c1i,c2i,…,cki}, and the union of all concepts related to those in the user profile can be expressed as R=⋃ciu∈ur(ciu). Hence, the extended user profile uR is obtained by extending the initial set of concepts extracted from the previously read articles with the set of related concepts, namely uR=u∪R.

Another assumption underlying RSR [78] is that the more articles containing concept ciu a user reads, the higher his interest in that concept. The weight wiu of concept ciu is thus defined as the number of articles from the user’s reading history that contain the concept. RSR uses a rank matrix – rows contain concepts from the initial user profile and columns denote the concepts in his extended profile – to model the interaction between concepts and compute their importance for the user. The rank of a concept ci,ju from this matrix is obtained by weighting wiu using an experimentally determined constant value meant to capture the type of relationship between concepts:

The final rank of every concept in the user’s extended profile, denoted Rank(ej)=∑i=1qrij, is computed as a sum of the values in the corresponding column in the rank matrix and stored in a vector vu.

A min-max normalization is applied to the extended user profile to ensure that the ranks are in the range [0,1], and thus, comparable between the user’s and article’s profiles. A news article, comprised from a set of concepts, is also represented as a vector of concept ranks vv, where a concept contained in the user’s extended profile and appearing in the unread article is assigned the same rank as in vu, while one not occurring in the target item has a rank of zero. Lastly, the extent to which an article is relevant to a user is computed as the ratio between the sum of concept ranks from the article’s representation and the sum of concepts ranks in the user’s profile:

The Ranked Semantic Recommendation 2 (RSR 2) [55] model improves RSR by considering, in addition to the concepts appearing in the unread news articles, also the concepts related to them. Following the previous example, this means that if a candidate news contains the concept instance Elon Musk, the model will also use related concept instances such as Open AI, SpaceX, Tesla, Inc., or Neuralink to represent the article. Thus, the original article profile is extended by the set of related concepts, namely vE=v∪E, where E=⋃cin∈vr(cin).

Another difference to RSR is that RSR2 uses different weight values to determine the concepts’ ranks. The rank of a concept in the extended article representation vvE is equivalent to the corresponding concept rank from the extended user profile vu, if it appears in it or is related to one of its concepts. Otherwise, a concept has a rank of zero. The final similarity measure between the extended article and user profiles is modified to incorporate these changes accordingly.

6.1.3.Summary

Non-neural, entity-centric news recommendation techniques are summarized in terms of three aspects:

6.2.1.Overall framework

The majority of methods in this category represent a news article as a set of tuples consisting of the concepts contained in an ontology and their corresponding weights. Formally, this can be written as v={⟨c1n,w1n⟩,…,⟨cpn,wpn⟩}, where cin∈O, win is the weight of concept cin(1⩽i⩽p), and p is the total number of concepts found in the article v. The profile of a user u is constructed by accumulating all the concepts that appear in the articles previously read by the user, denoted as u={⟨c1u,w1u⟩,…,⟨cqu,wqu⟩}, where wju is the average weighting of concept cju in the articles from the user’s reading history that contain concept cju, and q denotes the number of concepts in the articles read by the user. The recommendations are nearly always computed in a content-based, item-ranking fashion. We distinguish non-neural, path-based frameworks in terms of three aspects:

6.2.2.Representative models

In the following, we investigate six representative recommendation techniques for this category.

ePaper [105,148] weighs the ontology concepts denoting the user’s interests according to the user’s implicit feedback. More specifically, the weight of a concept ciu is given by the number of clicks on articles containing the given concept relative to the total number of clicks in the user’s profile. The relevance of an item to a user is defined in terms of the hierarchical distance between the concepts from the associated profiles, which takes into account the amount of common and related concepts included in each profile, as well as the distance between them. Based on a 3-level ontology, ePaper relies on 1-hop (parent-child) and 2-hop (grandparent-grandchild) hierarchical relations between concepts [105]. The relative position of related concepts from the user’s and the article’s profiles denotes their relationship in terms of specificity.

Three types of partial matches between concepts were defined by Maidel et al. [105] based on hierarchical distance. A perfect match is obtained if the same concept appears in both profiles and at the same hierarchical level. For example, both the news and the user profile contain the concept ‘artificial intelligence’, found at level 1 in the ontology. However, if a concept occurs only in one of the profiles, while its parent or child is included in the other profile, a close match is reached. In this case, one can further differentiate between cases when the user’s concept (e.g. artificial intelligence) is more general than the article’s concept (e.g. deep learning), and those in which the user’s interest is more specific (e.g. user concept is graph neural networks and item concept is deep learning). Lastly, a weak match occurs if the concepts from the two profiles are two levels apart in the hierarchy, such as the user being interested in graph neural networks, whereas the article contains the concept artificial intelligence. Analogous to the previous match type, two cases are determined by the profile containing the more general concept.

A similarity score Si assigns different weights based on the type of match of concept c1w to the corresponding concepts in the user’s profile. Lastly, the Item Similarity (IS) score, shown in Eq. (29), determines how similar the target article is to the user’s interests, based on the number of concept matches (given by Si) and the concepts’ weights from the user profile, given by the number of clicks N on the items containing the concept [148].

A different approach is adopted in Magellan [45], which uses a Weighted Term Frequency scheme to determine the importance of a candidate news article to a monitored domain. Magellan extracts named entities from news to represent the articles and operates on their corresponding concepts from an ontology. According to the weighting scheme, the importance of concepts is determined by their centrality and prestige [99] in the ontology. The main assumption underlying the measure of centrality is that the more relations a concept has to other concepts, the higher is its importance in the given domain. Hence, the concept with the highest out-degree, namely the largest number of accumulative out-going connections, is considered the top-ranked individual. Subsequently, the importance of the remaining concepts depends on the distance, measured in the number of hops, and the strength of the relations wr to the top-ranked concept, as given by the centrality weight wcentrality=1hops×wrhops.

The centrality score ensures that concepts with shorter and stronger connections to the top-ranked concept will be assigned higher importance than those situated further away in the ontology or having weaker relations. The centrality weight is complemented by the prestige of a concept in the ontology, a method that ranks the concepts based on their incoming relations. The more a concept is referred to via different relations by another concept (i.e. the larger its in-degree), the higher its prestige in the ontology. Consequently, the final importance score of a concept is computed as the product of centrality and prestige (denoted as rank in Eq. (30)), weighted by a constant value α assigned to the top-ranked concept:

The final weight wi of concept i is obtained by combining its importance in the ontology and frequency ni in the news article, w=wimportance×ni. According to this weighting scheme, Magellan will assign higher scores to news articles which frequently contain entities with large importance in the ontology, whereas those which either contain few concepts or only named entities with low importance will be assigned a lower score.

Similar to SF-IDF, the Semantic Similarity (SS) recommendation model [26] represents a news item using the WordNet synsets of the terms it contains, as shown in Eqs (10) and (11). Recommendations are generated by comparing the similarity of the synsets in the unread news article to the synsets of all the articles previously read by the user. For this purpose, firstly a vector containing all combinations of synsets from the target article and the union of synsets from the user profile is constructed as follows:

Furthermore, a subset is created from V for all pairs of synsets sharing the same part-of-speech (POS):

The final similarity score of an unread article is given by the sum of all combinations’ similarity rank sim(sn,su) relative to the total number of combinations |W|, illustrated as follows:

The WordNet taxonomy constitutes a hierarchy of “is-a” relationships between its nodes which, in turn, constitute synsets. As such, Capelle et al. [26] propose five semantic similarity measures to calculate the similarity rank sim(sn,su) for each combination of synsets in W, namely the extent to which two synsets are semantically close. Three of the measures (Jiang and Conrath [82] simJ&R, Resnik [135] simR, Lin [97] simL) utilize the information content of a node, defined as IC(s)=−log∑w∈Sp(w), where p(w) denotes the probability of instance w of the synset to appear in the corpus. More specifically, this metric can be described as the negative logarithm of the sum of all probabilities of all the words w from synset s. Furthermore, they take into account the lowest common subsumer (LCS) between two nodes, which represents the lowest node dominating the pair [135]. The three metrics are illustrated in Eqs (34)-(36).

The two remaining metrics, of Leacock and Chodorow [90] simL&C, and of Wu and Palmer [184] simW&P, shown in Eqs (37)-(38), define the similarity based on the path length between nodes. The path length can refer to either the shortest path (denoted length) between a pair of nodes or the maximum depth (denoted as D) from the least common subsumer to the top node in the hierarchy.

Similar to Bing-SF-IDF+, BingSS [27] extends the semantic lexicon-driven SS recommendation model by taking into account named entities. The semantic similarity formula from Eq. (33) is modified to take into account only the set of synset pairs TOPWβSS with the highest similarity in W, as follows:

Lastly, the Bing and the SS components are combined in the final BingSS similarity score using a weighted average with predefined weight α:

OBSM, the ontology-based similarity model proposed by Rao et al. [130], uses a TF-IDF weighting scheme for the concepts in the user and news profiles. The similarity between two concepts c1 and c2 found in the news depends on their ontological structures, represented in terms of the shortest distance d among concepts in the ontology, the shortest distance δ to their common ancestor closest to the root node, and the height H of the ontology. This concept-concept similarity metric, illustrated in Eq. (42), follows the assumption that two adjacent, more concrete concepts situated at a lower level in the ontology share more common information from their ancestors, and thus, have a higher likelihood to be similar than those found at a higher level. The preference for closer concepts is ensured by the term (−log2Hd2H), which is negatively correlated with the concept distance d. In turn, the weight eδeδ+1 will assign higher importance to concepts located at deeper levels in the hierarchy.

The similarity between the profile of a target news article and user is computed in the following way [130]:

According to Eq. (44), two concepts cin and cju whose corresponding weights win and wju are relatively equal, will result in a higher confidence score wi,j. In turn, this means that the two concepts are similarly important in their concepts sets, indicating that the target article might be of interest to the given user. Concepts with different weights in their associated sets are penalized using a smoothing factor k which controls the sensitivity of the confidence function.

In contrast to the previous models, SED, the entity shortest distance over knowledge graphs algorithm proposed by Joseph and Jiang [83], defines item-item similarity as the shortest distance between the subgraphs consisting of named entities extracted from news articles. The approach is threefold [83]. Firstly, all named entities contained in every news article are extracted and linked to the corresponding nodes in a knowledge graph. Secondly, in the subgraph generation phase, each news article is represented as a subgraph containing the linked nodes from the knowledge graph associated with the previously extracted named entities. These subgraphs are expanded with outgoing relations from the L-hop neighborhood of each node discovered using a breadth-first search strategy.

The shortest distance between two entities over the knowledge graph represents the shortest path length between the corresponding nodes, mathematically denoted as D(ei,ej)=min(|pk|), where |pk| is the length of path k from the set of all paths between the entity pair (ei,ej), namely pk∈P(ei,ej). Based on this definition, the shortest distance between two articles’ subgraphs, S1 and S2, is computed according to Eq. (45).

Lastly, the similarity between the two articles is computed as the pair-wise shortest distance over the union of their subgraphs [83], as shown in Eq. (46).

This method provides a symmetric average minimum row-wise distance which places higher importance on the entity pairs with the highest likelihood of co-occurrence in news article. Additionally, a weighted shortest distance between the articles could be used by weighting the edges of the subgraphs and computing the sum of all the weights of the traversed edges [83]. For the weighted SED algorithm, different weighting schemes could be used, including the relation weighting scheme, which assigns edge weights based on the number of shared neighbors of two entity nodes from an article.

6.2.3.Summary

Non-neural, path-based knowledge-aware news recommender systems are summarized from the following perspectives:

6.3.1.Overall framework

Frameworks classified in this category generally use a knowledge distillation process to incorporate side information in their recommendations. Firstly, named entities are extracted from news articles using a named entity recognizer. Secondly, these are connected to their corresponding nodes in a knowledge graph using an entity linking mechanism. Thirdly, one or multiple subgraphs are constructed using the linked entities, their relations, and neighbors from the knowledge graph. Afterwards, the obtained graphs are projected into a continuous, lower-dimensional space to compute a representation for their nodes and edges. Thus, these models use both the structural and semantic information encoded in knowledge graphs to represent news. Figure 3 exemplifies this process.

Fig. 3.

Illustration of the knowledge distillation process used by neural-based recommendation models (reproduced from [170]).

In contrast to models from the previous categories, the neural-based recommenders we reviewed for this survey predict the probability that a user will click on a target article, namely the click-through rate. We consider several factors underlying these recommendation models:

6.3.2.Representative models

The architectures of 11 neural-based news recommendation frameworks are discussed in this section.

The Collaborative Entity Topic Ranking (CETR) [193] model combines matrix factorization, topic modeling, and knowledge graph embeddings in a collaborative fashion to alleviate the data sparsity problem and the limitations of word-level topic models on very infrequent words appearing in news articles. The model joins together three modules, the first modeling the user’s reading behavior, the second performing entity-level topic analysis of news, and the last computing representations of the knowledge graph entities.

The user behavior component takes as input the user-news interaction matrix Y, defined as follows:

The user-item interaction matrix is factorized into a matrix U of user features and a matrix V of news latent features. The factorization method, a Bayesian Personalised Ranking (BPR) model [134], uses a sigmoid function to characterize the probability of observing a triplet (u,v,v′) given the user and news matrices. Such a triplet denotes the scenario in which a user u has read article v, but not v′. The two feature matrices are learned with a maximum likelihood function applied over all triplets in the user’s profile.

In the following step, topic analysis is conducted at the entity level, where entities belonging to the same topic are sampled from a Gaussian distribution [193]. The third module learns knowledge graph embeddings with the TransR model [98]. The probability of observing a quadruple (h,r,t,t′), denoting the head entity h being connected to tail entity t, but not to t′, by relation r, is defined similarly to BPR. The three components are jointly trained by calculating the log-likelihood of seeing all triplets, entities, and quadruplets, given the user and news feature matrices, the distribution of topics over entities, and the embeddings of entities and relations from the knowledge graph.

DKN, the deep knowledge-aware network proposed by Wang et al. [170], was the first architecture to fuse neural network-based text-level and knowledge-level representations of news using an attention module. The input to the recommendation model is constituted by the user’s click history and one candidate news article. Each article t is represented by its title. In turn, the article’s title is composed of a sequence of words, t=[w1,w2,…,wN], and every word w might correspond to an entity e in a knowledge graph [170]. The enrichment of textual information with external knowledge follows the knowledge distillation process from Fig. 3. Wang et al. [170] use not only direct knowledge graph correspondents of identified named entities to construct the subgraph, but also their one-hop neighbors to reduce sparsity and increase diversity among the extracted entities. This knowledge-level representation of news is further enhanced by taking into account the context of an entity, denoted as context(e), to increase the identifiability of entities after computing their embeddings.

The inner-circle in Fig. 4 exemplifies this concept. DKN takes as input the embedding of GameStop short squeeze to represent the entity, as well as its context, denoted by neighbors and associated relations, such as USA (country), or Elon Musk, Robinhood, r/WallStreetBets (participant).

Fig. 4.

Illustration of ripple sets of GameStop short squeeze in Wikidata. The concentric circles indicate ripple sets with different hops. The fading blue signifies decreasing relatedness between the center and the neighboring entities (reproduced from [168]).

One of the input elements to the recommendation model is constituted by the embedding of an entity’s context, defined in the following manner.

The first level in DKN’s architecture is represented by a knowledge-aware convolutional neural network (KCNN), namely the convolutional neural network (CNN) framework proposed by Kim [85] for sentence representation learning extended to incorporate symbolic knowledge in the text representations. Firstly, the entity embeddings ei∈Rk×1 and the context embeddings e¯i∈Rk×1, obtained with TransD [80], are projected from the entity to the word vector space, according to Eqs (50) and (51), using a hyperbolic tangent transformation function g.

Secondly, the matrices containing word embeddings w1:n (pre-trained or randomly initialized), transformed entity g(e1:n) and context g(e¯1:n) embeddings are aligned and stacked to obtain a multi-channel input W=[[w1g(e1)g(e¯1)]...[wng(en)g(e¯n)]]∈Rd×n×3.

The word-aligned KCNN applies multiple filters of varying sizes to extract patterns from the titles of news, followed by max-over-time pooling and concatenation of features to obtain the final representation e(t) of an article. Hence, the KCNN component is able to discover latent knowledge-level connections among news using extracted entities and common sense knowledge embedded in knowledge graphs.

Additionally, DKN employs an attention network to capture the diverse interests of users in different news topics by dynamically aggregating a user’s history according to the current candidate article [170]. The second level of the DKN framework concatenates the embeddings of a target news tj and an article tk read by the user, feeding the resulting vector into a Deep Neural Network (DNN) H which computes the impact of the candidate news on the read article. The output of the attention network H is normalized using a softmax function. This process is illustrated in Eq. (52):

Given the normalized attention weights, a user i’s embedding with respect to the target article tj is represented by the weighted sum of the Ni embeddings of article titles from his click history:

Lastly, DKN [170] predicts the click probability of user i for news article tj with another DNN G that takes as input the final user embedding from Eq. (53) and the article’s embedding, as pi,tj=G(e(i),e(tj)).

The recommendation model of Gao et al. [58] learns semantic-level and knowledge-level representations of news by adjusting the DKN architecture to use a fine-grained word-level description of news, obtained with a self-attention mechanism, instead of a topic-level representation given by the KCNN component. The user’s click history and a candidate piece of news constitute the model’s input. The framework consists of four-level self-attention modules [58]. Firstly, a word-level self-attention component computes the semantic-level and knowledge-level representation of articles using pre-trained embeddings of news tags and transformed pre-trained embeddings of entities extracted from a knowledge graph and their context, similar to DKN. The attention weight measuring the impact of each word in the news representation is computed as follows:

Secondly, the item-level attention model computes the final representation of news article tk, according to Eq. (55), as a weighted sum of the different-level embeddings, where the weights are given by corresponding attention coefficients.

The attention weights of words are calculated as shown in Eq. (56), while those of entities and context can be computed analogously.

Thirdly, the user-level self-attention module computes the final representation of the user i’s history e(i) with respect to the candidate news tj as in Eq. (53). However, in contrast to DKN, here the attention weight is computed as follows:

Fourthly, the vector representation of the user and the target news article are combined using a multi-head attention module [163] with ten parallel attention layers. Lastly, the output of the fourth module is passed through a fully connected layer to calculate the user’s probability of clicking the candidate article.

A different approach is constituted by RippleNet [168], an end-to-end framework that propagates user preferences along the edges of a knowledge graph. RippleNet takes as input a candidate news article and the user’s historical set of interests Vu, which act as seeds in the knowledge graph. The main idea underlying the model is that of ripple sets Suk, namely sets of knowledge triples situated k-hops away from the seed set Vu. The concepts of relevant entities and ripple sets are defined below.