Diverse data! Diverse schemata?

1.Diverse data

Let us first answer the question of what data diversity is or could be. Several different perspectives come to mind. (1) Data can come in different forms such as structured, e.g., relational database tables and statistics; unstructured, e.g., plain text or imagery; and semi-structured, e.g., data stored in JavaScript Object Notation (JSON) or the Hypertext Markup Language (HTML). These data are diverse because they require different technologies to understand, interpret, and draw inferences. For instance, mining features from news articles is at the forefront of modern business intelligence, while methods for mining relational data are well established.

(2) Data can come from various sources that differ in scope, themes, or data culture. For example, data about the same phenomena will differ significantly depending on whether it is collected and published by an official, authoritative source or collected by a community of volunteers. Combining data from such different data cultures is desirable due to their complementary characteristics. Typically, data from government agencies is homogeneous, well documented, stable, conforms to certain quality standards, and so forth, while volunteered data relaxes these attributes in favor of timely updates, increased coverage, and additional properties not collected otherwise [2]. Similarly, combining data that differ in scope, theme, and coverage is beneficial because it fosters a more holistic understanding. For instance, integrating remotely sensed imagery about wildfires with forecasts for the resulting smoke plumes and particle density observations measured from ground stations, jointly improves the prediction of health risks or damage to agricultural products [5]. Another well-studied success story of combining near-real-time volunteered information with authoritative sources is disaster mapping and management [4]. For instance, volunteers update transportation infrastructure datasets using up-to-date satellite imaginary to quickly determine which roads have been damaged and how this affects overall connectivity within the transportation network, and, thus, improves disaster relief [6].

(3) Finally, data can be diverse because they describe the same phenomena and the same (observable) properties but provide a different perspective. For instance, data about the cradle-to-grave environmental footprint of electric cars may differ depending on the perspectives taken by research teams, industry sectors, governments, and so on. The definition of “electric cars,” the cutting lines of “cradle” and “grave,” the environmental impacts to be considered and so forth, are all subject to the observer’s perspective. Similarly, the environmental footprint and resource-efficiency of nations can be measured in many different ways [16] without implying that one measure is necessarily better than others.

2.Diverse schemata?

Interestingly, while the first two types of data diversity are unquestioned and have been part of data science theory and practice for years, this last type of diversity is often misunderstood and invites controversies. Intuitively, as scientists, we are inclined to believe that one perspective is more accurate, less biased, simpler, leads to better predictions, and so forth. To some degree, the idea of several equally-valid perspectives seems alien to us, maybe because of seemingly colliding with the law of excluded middles by which two statements that disagree cannot both be true.

Similarly, we tend to believe that data are raw [15], i.e., that they are independent of who observes. However, this is not always the case and particularly not for categorical data. For example, there is no ‘true’ definition of poverty, gender, forest, or planet, yet these terms play a prominent role in science and society. While it seems easy to claim that only the first two are partially defined by culture and society, the same is true for the latter examples as it is evident from the more than 600 commonly-used (and contradictory) definitions of forest [1, 9] and the changes [14] to the category of planets over time.

Put differently, many concepts are cognitive artifacts, and there are many ways to construct them.11 This leads to semantic heterogeneity. A key question that arises from this discussion is whether this semantic heterogeneity is a problem to be overcome [9] or a reality of Web-scale knowledge representation systems that work on the level of statements, not facts [7]. The first stance often calls for the standardization of meaning in the form of upper-level ontologies (or one common ontology [3]), while the second stance prescribes modular ontology design with a focus on common patterns and alignments between ontologies [13].

It is interesting to examine how disciplines that cannot afford the crisp nature of top-down axiomatic knowledge representation address this discussion. For instance, in machine learning and representation learning, data diversity is desirable during training to ensure that the resulting model captures the entire range of cases that it will encounter in the wild. Similarly, recent work on linguistic embeddings can distinguish between different meanings that terms take depending on the context [12]. Another example is Cognitive Science, in which many different theories of categorization are studied, including concepts with multiple prototypes [10].

From an even more abstract stance, the ongoing cultural goal of increasing workforce diversity is rooted in the assumption that diversity improves representation. Put differently, who we are, and where we come from (culturally and geographically), influence how we experience, i.e., categorize, the world around us.

This has important consequences for both the schemata we design and the representations we learn. Intuitively, increasing the number of classes to be distinguished reduces the accuracy of a model (while keeping other parameters such as training size invariant). Similarly, there are TBox axioms that are easy to learn by rule mining from existing knowledge graphs, but would fail to capture the context of data when schemata do not match how data exists in reality for lack of diversity in their construction [8]. Finally, for some concepts, we may even end up in situations where the features that can be extracted from a given source, e.g., a facial image, can no longer be used effectively for a task at hand, e.g., classification.

The resulting dilemma can be summarized nicely by the following observation: Diverse data and application needs require diverse schemata, while interoperability and integration benefit from common schemata. So, how do we support diverse (even contradictory) schema knowledge while avoiding another Tower of Babel? Many potential solutions were discussed in classical AI literature some decades ago, such as the notion of contexts and microtheories [11]. However, they do not answer how much diversity across schemata we deem to be beneficial, nor how to strike the right balance between the increasing complexity of individualized schemata and the need for efficient retrieval and integration. In fact, efforts such as Schema.org, seem to favor single, shallow vocabularies to fulfill application needs. Modular ontology design supported by structural patterns and expressive alignments between ontologies is another path forward [13]. Despite success stories, a large-scale, industry-strength application of these ideas is still missing.22

However, this is not a technical paper but one to start an important discussion: How diverse do we want our schemata to be and which price are we ready to pay in terms of prediction accuracy and reduced interoperability?