SAREF4health: Towards IoT standard-based ontology-driven cardiac e-health systems

3.1.ETSI smart applications reference ontology (SAREF)

In 2013, the European Commission launched a standardisation initiative together with ETSI in close interaction with the smart appliances industry to create an interoperability language for the various smart and networked devices from different manufacturers that co-exist in our homes. This initiative resulted in the Smart Appliances REFerence ontology (SAREF)77 (Daniele et al., 2015), which was developed with the smart home market (Daniele et al., 2016) and subsequently published as an ETSI Technical Specification.88 At first, this ontology was built as a reference model targeting smart appliance solutions for the energy efficiency optimisation in the smart home domain, e.g., from lamps and consumer electronics to white goods, such as washing machines and ovens. However, SAREF has evolved to cover more vertical domains and the IoT domain in general to foster cross-domain interoperability. For example, it should be able to define that a motion sensor provides the same functionality whether it is located in a smart home or in a street in the context of smart cities/smart mobility. Therefore, this ontology has been recently renamed as Smart Applications REFerence ontology (SAREF) to stress that it is not limited to the smart appliances domain, but it rather improves the integration of semantic data from and across various applications in different domains. SAREF provides building blocks that enable re-utilization of different parts of the ontology according to specific requirements. It is grounded on 47 “semantic assets”, i.e., standards, proprietary data models, protocols and other ontologies, including SSN.

The motivation behind SAREF was that the market would continue to be fragmented and powerless without a (protocol-independent) semantic layer that could enable interoperability among the various smart appliances from different manufacturers. To that end, SAREF was created with the intention to interconnect different platforms, supporting data exchange with different protocols. Over time, SAREF has evolved with the feedback of its users, some incorporated in SAREF 2.0.99 Current work in ETSI includes a release of SAREF 3.0, which is expected to be published by 2020. Some of the core elements of SAREF 2.0 are depicted in Figure 1.

As described before, one of the assets used by SAREF development is SSN (version 1.0), which inspired the definition of the main elements of SAREF, namely Device, Sensor, Unit of Measure and Time/Duration (see Figure 1), according to the high-level mappings provided in the SAREF initial documentation (Daniele et al., 2015). A saref:Device (e.g., a saref:Sensor) represents a tangible object designed to accomplish one or more functions in diverse types of locations (e.g., households and buildings). For example, a saref:Sensor saref:hasFunction saref:Function of type saref:SensingFunction. The SAREF ontology offers a list of basic functions that can be combined to define more complex functions and assign them to a single device. For example, a saref:Switch can offer an saref:Actuating function of type “switching on/off” and a saref:SensingFunction of type saref:LightSensor, so if there is illumination in the environment then the switch turns off the light. Each saref:Function has some associated saref:Command(s), which can also be selected as building blocks from a list. For example, “switching on/off” is associated with the commands “switch on”, “switch off” and “toggle”. Depending on the saref:Function(s) it accomplishes, a device can be found in some of the corresponding saref:State that is also listed as building block.

Fig. 1.

Core elements for representing device measurements in SAREF.

The composition (mereological) pattern of a saref:Device is represented through the saref:consistsOf self-relationship with the axiom saref:consistsOf only saref:Device. The saref:consistsOf element is defined as the “relationship indicating a composite entity that consists of other entities”, which is similar to ufo:componentOf, i.e., a meronymic relation that conceptualises part-whole. For example, the WM30 wind sensor (a saref:Device) can be defined as a composition of wind direction and wind speed sensors. A saref:Device measures a specific property, represented by the object property saref:measuresProperty to a saref:Property. For example, a saref:SmokeSensor (a saref:Sensor) measures saref:Smoke (saref:Property). Analogously, a saref:WindSensor measures saref:Wind. SAREF represents a measurement observed by a sensor in time through the saref:makesMeasurement object property of a saref:Device to saref:Measurement, representing the relation between a device and the measurements it makes, ontologically equivalent to sosa:Observation. A saref:Measurement element is related to its saref:UnitOfMeasure and the saref:Property measured.

3.2.SAREF extensions

One of the main benefits of SAREF is its extensibility for vertical markets, which is enforced by an ETSI standard procedure for extensions (EC, 2017). A number of SAREF extensions have been created over the years, starting from the Energy, Environment and Building domains1010 and resulting in the recently published extensions for the Smart Cities, Industry & Manufacturing and AgriFood domains. To avoid semantic interoperability issues when extending SAREF, ETSI defined an approach to be followed to create and submit a SAREF extension for acceptance. This procedure is based on the following best practices: (1) the extension is designed according to clarity, coherence, extensibility, minimal encoding bias and minimal ontological commitment criteria; (2) relevant stakeholders in the domain of interest should be involved in its development process; (3) the group/community that creates the extension should be committed to contribute to its maintenance; (4) it should not add concepts that are already present in SAREF or other extensions; (5) it needs to be properly documented and published.

Among the various extensions, SAREF4envi1111 is an extension to the environment domain and light pollution developed in collaboration with domain experts from the STARS4ALL H2020 project.1212 SAREF4envi introduced frequency-related classes and attributes like saref4envi:FrequencyMeasurement, so that a Device (saref:Device) has a frequency measurement (saref4envi:hasFrequencyMeasurement relation), which is measured in (saref:isMeasuredIn relation) saref4envi:FrequencyUnit unit, a SAREF unit of measure (saref:UnitOfMeasure type) that can be expressed in Hertz or reciprocal (second, hour, day, year).

Current ETSI efforts include the development of SAREF extensions for the automotive (SAREF4auto), e-Health and aging-well (SAREF4ehaw), wearables (SAREF4wear) and water (SAREF4watr) domains. These extensions are developed in the context of an EC-funded Specialist Task Force, called STF 5561313 that has started in December 2018 and will run for a period of two and a half years.

3.3.Semantic models for healthcare systems

A well-founded ECG ontology (UFO ECG) was designed based on an ontological analysis of existing health standards and ontology-driven conceptual modelling with UFO (Gonçalves et al., 2009). The main goal of UFO ECG is to serve as a reference “unified Electronic Health Record (EHR) model”, providing mappings to the three most common standards that support the representation of ECG data, including the HL7 annotated ECG standard (HL7 aECG) (Gonçalves et al., 2011). The use of HL7 v3 along with IEEE1451 in embedded devices to achieve end to end semantic interoperability on health systems is exploited by (Cosío-León et al., 2018). HL7 aECG (HL7, 2005) is one of these standards, which was chosen by the US Food and Drug Administration (FDA) for clinical trials, implemented as a common lexicon approach, i.e., using XML schemas, running nowadays in several hospital information systems. A review of ECG storage formats is given by Bond et al. (2011), which also includes SCP-ECG and DICOM. This review concludes that aECG inherits the verbosity of XML, since it produces large messages (25 times larger than a compressed approach), so that it may only be used in clinical drug trials and presents conceptual design issues that hinder its application in real life situations.

HL7 Fast Healthcare Interoperability Resources (FHIR) is an emerging open standard that is considered as the evolution of the HL7 standards for the new generation of health data exchange. HL7 FHIR defines a collection of “resources” (data model elements) that can be mixed and adapted for particular clinical contexts through the “profiling” process. Together with W3C semantic web community, the HL7 community developed an RDF representation of the FHIR data model, which can be described and validated with Shape Expressions (ShEx) (Solbrig et al., 2017).

The central FHIR element is the Observation resource,1414 which is used to represent diagnosis and monitor progress, and is recommended for monitoring vital signs and device measurements. In particular, the “EKG example using Sampled Data”1515 demonstrates how to serialise ECG data using the Observation resource, which offers the property valueSampledData, capable of representing the ECG sample sequence as a series of measurements of the heart electrical activity through the SampledData element, similarly to HL7 aECG. A SampledData provides a concise way to handle the data produced by devices that sample a particular physical state at a high frequency, typically used to represent the output of an ECG device. The data type includes a series of raw decimal values, which are mostly simple integers, along with adjustments for scale and factor.

The Open Biological and Biomedical Ontology (OBO) Foundry is a popular ontology-driven initiative in the biomedical informatics field, which provides a number of domain ontologies leveraged by the Basic Formal Ontology (BFO), a foundational ontology. For example, OBO provides the Eagle-I Resource Ontology, which includes several terms for health-related instruments and protocols, such as ECG monitoring device.1616 SNOMED CT1717 and LOINC1818 are other relevant ontology-based approaches for clinical health terminologies, since they define terms like ECG voltage and ECG abnormal finding.

Finally, ETSI SmartBAN TC1919 has proposed and standardised a reference model and associated modular ontology for Body Area Networks (BANs) and medical devices (sensors, actuators, wearables) interoperability management at all levels (data/informational/semantic level, device/technical level and network level) (ETSI, 2019a; 2019b). At technical/informational/semantic levels, the proposed solution relies on the specification and formalisation of both: (1) a BAN devices/data (measurements and control/monitoring data included) unified semantic and open reference model, associated with corresponding unified metadata and reference modular ontology, which unfortunately does not consider the time series concept; and (2) a BAN reference service model and associated service ontology. This service model, coupled with a WoT strategy, in particular handles semantic interoperability, as device discovery and composition features, and cross domain use cases. At network level, the proposed solution relies on the specification of generic Multi-Agent/oneM2M based enablers integrated into a global IoT reference architecture dedicated for secure interaction and access to BAN data and entities. This reference architecture is specified on top of the two aforementioned BAN reference models and has been clinically tested for an ‘elderly at home support and monitoring’ use case in the context of the CareWare ITEA3 project.2020

4.1.General challenges

We identified that ontology-driven IoT healthcare systems usually require conceptualizations to be matched and multiple domain ontologies to be combined, because of the cross-disciplinary nature of the healthcare field, in order to provide a higher level of interoperability for systems that perform data fusion from multiple sources. Semantic Gateways address this issue by enabling the execution of semantic translations that can tackle specific semantic integration problems, such as semantic overload, ambiguity and information distortion (Kim et al., 2012).

Semantic translations depend on mappings between two semantic models, which guide how the input data represented with the source ontology can be transformed to data represented with the target ontology, similarly to metamodeling-based transformations in Model-Driven Engineering (MDE). For example, the specification of bi-directional transformations between EDXL-TEP and HL7 v22121 provides a set of mappings regarding the patient concept, albeit EDXL-TEP and HL7 v2 are not semantic models. SSN ontology and several other ontologies in the context of IoT, e.g., ETSI SAREF, provide mappings to OGC SWE, especially SensorML and O&M, enabling higher interoperability among these standards. In the past decade, new interoperability standards for sensor systems have been developed, including lexicon and semantic models, as well as proprietary data models, resulting in a number of different data representation assets. Some projects incorporated these standardised ontologies, but few of them addressed their alignment to support sensor data integration. To the best of our knowledge, approaches based on Semantic Gateways that support the whole development lifecycle of semantic translations, i.e., specification and implementation, considering their configuration at runtime, have not been exploited in any IoT healthcare system.

Our ambition has been to improve the semantic interoperability of IoT healthcare systems by considering not only IoT ontologies, but also addressing cross-domain relationships through the extension of a standardised IoT ontology with alignments to multiple domain-specific ontologies. Therefore, our research goal has been to improve the semantic integration of components of an IoT healthcare system, enabling seamless integration with other systems. We identified the following challenges to achieve this goal, along with the corresponding requirements:

4.2.Study case requirements and use cases

The INTER-IoT project defined a scenario that aims at decreasing the risk of fatal accidents at the port of Valencia, improving health prevention and enabling quick reaction by reducing response time. This scenario requires integration between logistics and e-Health domains. The goal was to exploit how cardiac e-Health can use IoT platforms dedicated to logistics to prevent the occurrence of accidents and to support evacuation or attention in case of emergency situations (Moreira et al., 2018c).

The main actor involved in this scenario is the truck driver, who works for a haulier company providing transportation of goods to/from the port. This company has an IoT platform that tracks the trips made by the trucks and monitors the health of the drivers. The solution addressing the requirements must be a third-party application for monitoring the data provided by the haulier IoT platform, integrating logistics and health data, providing the emergency notifications to be consumed by the port emergency command centre. This emergency centre is simulated in two ways: (1) an User Interface (UI) based on Google Maps, able to plot the data in real-time; (2) a third-party partner provides an emergency management system that consumes data from the port IoT platform and integrates them with the emergency notifications, being able to coordinate the emergency response.

INTER-IoT-EWS (Moreira et al., 2018c) is an IoT-based Early Warning System (EWS) developed to detect emergencies and risks of accidents with truck drivers wearing the Shimmer3 ECG unit2222 while they are transporting goods in a port area. Non-functional requirements include the semantic integration of IoT assets, e.g., devices, gateways, brokers and applications. Functional requirements include the real-time detection of arrhythmia (bradycardia and tachycardia) from ECG data provided by the ECG device and the detection of vehicle collision/impact by processing the cross-axial function (x2+y2+z2) of instantaneous tri-axial acceleration data, similarly to Fall-MobileGuard (Fortino and Gravina, 2015). Functional and non-functional capabilities are:

(IIOT-FC1) IoT platforms should be able to coordinate with emergency systems by detecting accidents and risks with trucks within the port area. The EWS should be able to identify vehicle collisions and severe changes of the driver’s cardiac behaviour, alerting their urgency and severity to multiple targets. The acceptance criterion is to check if the EWS, built on top of IoT platform(s), is able to coordinate with emergency systems through emergency interoperability standard(s).

(IIOT-FC2) The haulier IoT platform and the port IoT platform should be able to share cardiac information about the driver, monitored in real-time through an electrocardiography (ECG) device. The solution should be able to provide both raw and calculated data, e.g., ECG sequence (time series) and heart rate (HR). These data need to be integrated in a way that the port emergency control system can consume them. The acceptance criterion is to check if the EWS is able to process health data at the application level to identify cardiac issues and send data to emergency systems based on emergency interoperability standard(s).

(IIOT-NFC1) IoT platforms should be semantically and syntactically interoperable. The solution should be able to integrate the involved IoT platform(s) so that their data syntax and semantics are understandable, i.e., can achieve common understanding among the participating parts. The acceptance criterion is to check the use of a mechanism to translate data syntax (lexicon) and semantics of messages exchanged.

(IIOT-NFC2) E-Health and logistics should be integrated at the INTER-IoT application and semantics (A&S) level, including primitives for the interpretation of medical and logistics data. The acceptance criterion is to check the use of semantic models to represent e-health and logistics data within the IoT use cases.

(IIOT-NFC3) The energy consumption (battery level) of the devices being used for the situation identification mechanism should be monitored. The acceptance criterion is to check if the solution is able to provide energy consumption data for the application level, to be consumed by the EWS.

4.3.Comparison of semantic models for IoT-based ECG monitoring systems

Allowing the exchange of lightweight messages among medical sensors and/or wearables, gateways and cloud infrastructure is an important requirement of IoT solutions when processing big data. One of the main problems is that the verbosity of messages impacts the data exchange performance and cloud infrastructure costs. For example, the costs of the Microsoft Azure cloud gateway (IoT Hub) vary according to the message payload (“message meter size”).2323 A common solution to this problem is to aggregate measurement data at the gateway level according to a certain frequency, transmitting time series from the local gateway to the cloud from time to time (Fortino et al., 2014), as illustrated in Figure 2. The drawback of this approach is that when aggregating a series of measurements of one element, metadata about each specific measurement is lost, affecting the ontological expressiveness of the messages. Figure 3 illustrates this problem, showing how the ECG data represented with FHIR serialised as an array of numbers lose the details of each specific measurement, such as the timestamp, the unit of measurement and the property that the measurement is related to. Figure 3 also shows the representation of the aforementioned aspects of one measurement with SAREF.

Fig. 2.

Common approach to deal with high-frequency data in IoT applications.

Fig. 3.

Representation of a time series measurement with FHIR (left) and with SAREF (right).

UFO ECG represents series of measurements with the Sample sequence element, which is equivalent to the FHIR Sampled Data element. According to UFO ECG, Observation series represents the complex event composed by (a part-of relationship) Observations that are evenly spaced in time, which are carried out in a Recording session. A Sample sequence is a collective, i.e., an ultimate sortal, which has parts that play the same functional role in the whole, represented as an ordered sequence of data units that results from an Observation series. OpenEHR emphasizes this requirement of time series representation and includes the concept of “a history of events, i.e., as a time series, allowing all software to access data in a uniform way”. The root object (History class) provides a list of events as attribute.

Like ETSI SmartBAN, SAREF does not define a term for the time series concept, only allowing the representation of each granular measurement of a time series through the Measurement element. Although this approach improves the expressiveness of the data representation, thus the ontology quality, it produces an overhead to the message size when serialised. Therefore, a verbose message is generated when instantiating ECG data with SAREF, compared to the very same data represented with FHIR, UFO ECG or OpenEHR. For example, when comparing a JSON-LD message that instantiates SAREF individual Measurements against the similar message in FHIR RDF, but with SampledData element, our experiments on ECG data serialisation show that the SAREF message size is fifty times bigger (more verbose) than the FHIR message. This was calculated based on our performance evaluation (Moreira et al., 2018b), where an ECG device configured for 256 Hz frequency provides data to a semantic gateway, which accumulates data each 5 seconds before sending the message to the cloud, thus 1280 (5×256) measurements per message. While the equivalent SAREF message size has around 5 Mb, the FHIR message size has around 100 Kb. Even worse, the size difference between SAREF and FHIR messages grows exponentially with the number of measurements.

While in FHIR (or UFO ECG or OpenEHR) a measurement is only a number added to the data element (property of SampledData) representing data of the time series, in SAREF (as well as in ETSI SmartBAN) a Measurement is a data structure that implements a number of properties (e.g., isMeasuredIn and relatesToProperty). Similarly to SAREF, W3C SSN/SOSA does this through the Observation class, alleviating this issue by introducing the hasSimpleResult property, which enables the direct link to a number (literal), but still requires the representation of other properties (e.g., observedProperty). Therefore, due to this verbosity issue, we conclude that default SAREF is not suitable for exchanging IoT-based ECG time series data in real-time.

We studied alternatives, identifying their benefits and drawbacks. Table 1 compares the semantic models2424 considered in this study. The quality of the semantic models is the first characteristic we compared, following the best practices of ontology engineering described by Gyrard et al. (2015), more specifically the best practices #8 and #12, which regard common ontology pitfalls and ontology reuse, respectively. While UFO ECG, SAREF, SmartBAN and SSN/SOSA are built upon common conceptualization mechanisms, merging different concepts in the same class/property and reusing existing ontologies, FHIR RDF presents poor quality because it is a (semi) automatic serialisation of FHIR lexicon standard. Thus, FHIR RDF is a straightforward serialisation of the FHIR standard data model, which causes an overload of object properties, e.g., it has more than ten properties that represent the “description” concept, and lacks reuse of common terms, e.g., instead of having more than ten “description” terms, it could reuse dc:description.

The message size (payload) indicates whether the semantic model allows the representation of time series, as in UFO ECG and FHIR RDF. We also evaluated whether the semantic model is appropriate to the IoT context, i.e., whether it provides grammatical constructs for the main concepts of the IoT domain, namely sensor, actuator, protocol, observation, unit of measurement and measurement property. Specific characteristics of the ECG domain were considered in this analysis, such as the mereology of an ECG device, the elements participating in an ECG recording session, the ECG leads responsible for measuring frequency-based sample sequences (time series). Since our goal is the development of an EWS for the detection of risks of truck accidents, we also considered how to represent and retrieve multiple sensors data, such as acceleration data to calculate tri-axial function (impact) and ECG data to calculate ECG waveform features. As mentioned earlier, SSN/SOSA and SAREF are the most appropriate IoT reference ontologies, while some FHIR RDF resources can be used to represent some of the IoT constructs. Finally, we evaluated whether the semantic model is standardised by a standardisation body, such as HL7, ETSI and W3C for FHIR, SmartBAN/SAREF and SSN/SOSA (respectively).

In our comparison study, we concluded that none of the available standardised IoT ontologies provide adequate balance of quality and payload representing time series for ECG data. Therefore, the problem addressed in this paper is how to achieve these four characteristics in a single ontology, which led us to the development of SAREF4health as a SAREF extension.

The main information requirements of this ontology regarding ECG and IoT domains can be described as competency questions. While UFO ECG provides a high quality and deeper ontological analysis of the ECG domain, with rich descriptions following foundational categories, SAREF can be used as our IoT reference model of choice. Therefore, the set of competency questions shown in Table 2 has been defined both according to the main elements of the aforementioned reference ontologies and considering the emergency use case of the INTER-IoT project.

5.1.Design methodology

SEMioTICS follows a semantic Model-Driven Engineering (MDE) approach, giving emphasis to the semantic improvement at the conceptual modelling level and MDE transformations. The SEMIoTICS MDE methodology prescribes specification and implementation phases at design-time, considering the EWS deployment at runtime. The design-time level is organized in three interrelated parts guided by the model-driven approach:

The conceptual part supports the modelling language constructs of the specification part, which is mapped onto the appropriate technology implementations by our MDE process, where implementation is (partially) generated by MDE transformations. Horizontal (same abstraction level) exogenous (different languages) transformations are applied at the specification level to integrate models, while vertical transformations are applied to generate code, as described by Moreira et al. (2015a; 2015b). For example, SAREF4health was specified (designed) with support of OntoUML, along with the model validation approach, while the RDF implementation was (partially) generated from pre-defined MDE transformations from OntoUML to RDF (Moreira et al., 2016).

SAREF4health plays the role of core context model for the envisioned use cases, being developed as a well-founded core ontology. A well-founded core ontology provides a precise definition of a specific field described with an ontological language, being independent of a specific application (Scherp et al., 2011). OntoUML is an ontological language that provides structural and temporal predicates. In practice, OntoUML is an extension of the UML metamodel that uses stereotypes to represent UFO concepts. In order to design the SAREF4health extension, we applied the cyclic semantic enhancement process prescribed in the UFO research, increasing the quality of the core ontology through model assessment based on formal lightweight verification and validation activities (Moreira et al., 2016). The UFO ECG ontology played the role of a unified well-founded EHR reference model to represent the ECG domain. A benefit of this approach is that UFO ECG provides links to HL7 aECG, which FHIR is based upon, thus enabling straightforward alignments to FHIR RDF.

The SAREF4health design was leveraged by the same interactive and iterative approach experienced in the other standardised extensions of SAREF,2525 which was created in a transparent manner to allow stakeholders to provide input and follow the evolution of the work. The first step comprised the requirement collection to guide the implementation and validation of the ontology. Numerous information sources were analysed, as specifications, datasets, standards, APIs and data formats, as well as domain expert opinions and existing initiatives in the healthcare domain. The second step comprised the use case collection, specified in natural language. The third step comprised the purpose and scope definition of the ontology for the specific use cases regarding the monitoring of the cardiac behaviour of drivers, reflected in the competency questions in Table 2, in a way that abnormal situations, e.g., arrhythmia (bradycardia and tachycardia), could be detected by the emergency system.

5.2.ECG time series

The main definitions represented by the SAREF4health elements are described in Table 3. Although UFO ECG and the FHIR standard have been carefully designed in collaboration with healthcare experts, we decided to review the terminology used there. For example, we argued why UFO ECG uses the term Sample Sequence, while FHIR uses Sampled Data to describe the very same concept. Either way, the terms reflect the Time Series collective concept, i.e., a sequence of data units in successive equally spaced points in time, so that each sampled data unit plays the same role in the series (the ‘whole’).

In SAREF4health, we introduced the term Time Series Measurements to refer to a time series of a sequence of measurements made by a device, in line with the terminology often used in the measurement science (metrology).2626 We did not assign this term to the ECG context by prefixing it with ECG because it can be applied to other types of measurements. An ECG Sample Sequence is a Time Series Measurements that is measured in Electric Potential units (an array) and relates to the Heart Electrical Activity property. A Sample (UFO ECG) can be interpreted as a Measurement (SAREF), so we classify Measurement as a kind.

A crucial design decision was to classify Time Series Measurements as Measurement in SAREF, for two reasons: (i) this representation adheres to the definition of Measurement, i.e., measured value (Electric Potential units) of a property (Heart Electrical Activity); (ii) we reused the SAREF structure for class axioms of object properties, e.g., hasTimestamp, isMeasuredIn and relatesToProperty. The main implications of this choice are twofold: (1) ontologically this specialization is incorrect, since a collective cannot specialize a kind; (2) the hasValue property limits the value domain of a Measurement to exactly one float number. The hasValues property was added to overcome this issue, in which a Time Series Measurements can instantiate this property multiple times as an array of float numbers. In the conceptual model, instead of adding this specialization, the relation makesMeasurement between an ECG Lead and an ECG Sample Sequence makes this design decision explicit.

Finally, following the same approach of UFO ECG with Sample Rate data type and FHIR with period data property of SampledData, a class axiom was added in Time Series Measurements to relate it to a frequency (rate). For the definition of frequency, we reused SAREF4envi by importing the hasFrequencyMeasurement object property.

5.3.ECG device behaviour

An ECG device is usually referred as an ECG unit that plays the role of a recorder in the complex event (action) of an ECG Recording Session. In SAREF, we can classify this complex action as a Task that an ECG device accomplishes. Therefore, accomplishes plays the role of the inverse of the hasParticipant relationship (same as isAccomplishedBy). The hasParticipant relationship between ECG Recording Session and Person Under ECG Monitoring is represented with dc:author (Dublin Core).

In UFO, the event stereotype provides the relations start and end to limit the event temporal boundaries, as used in UFO ECG. SAREF defines the hasTime property, which is a “relationship to associate time information to an entity”, thus, we specialized this relation with hasStart and hasEnd, adding them as class axioms of Task. In UFO ECG, the other participant in this event is a Patient, but in SAREF4health we decided to generalize this ontological commitment, since someone does not need to be a patient (person under medical treatment) to participate in an ECG recording session. Thus, we introduced the Person Under ECG Monitoring element, i.e., a role of a Living Person, which is a phase often used in the UFO research (Guizzardi, 2005). This ontological commitment is motivated by the EWS use case, since a driver is not necessarily a patient. In SAREF4health, a Living Person is a Person, which is a term with similar definition in DOLCE Ultra-Light and schema.org. We only consider ECG for humans in this ontology.

Usually, the frequency of an ECG device can be set through an API, which becomes the frequency of each ECG Sample Sequence measured during a Recording Session. Therefore, we added a class axiom to the ECG device element: has Frequency Measurement property with range of only Frequency Measurements (from SAREF4envi, see Section 3.2). Although it seems redundant, this approach is required to differentiate the current frequency of a device from a frequency used in prior sample sequences. For example, the device Shimmer3 ECG can be set for sampling frequency (rate) of 512 Hz, which is recommended for clinical grade ECG, i.e., 512 data samples per second or an interval of 0.002 seconds between two consecutive data samples, from 0.05 Hz to 8000 Hz range. Suppose that after collecting some sample sequences and before sending the message to the gateway, the frequency is set to 256 Hz and new sample sequences are collected. With our approach, the message describes the current frequency of the device (256Hz) and the frequencies used in each collected sample sequence (512 Hz and 256 Hz).

Fig. 4.

SAREF4health main elements (green) with instantiated examples (blue).

5.4.ECG leads composition

An ECG Device registers the Heart Electrical Activity through electrodes attached to different places of the body, under the obvious assumption that the heart is beating inside the body of a living person. Two electrodes enable an ECG lead to be measured, which is an electrical vector characterized by the depolarization of the heart resulted by the electrical signal between the atria and the ventricles. Manufacturers commonly characterize an ECG device by its number of ECG leads. An ECG device is composed by extremity electrodes, which must be attached close to the left arm (LA), right arm (RA), left leg (LL) and the right leg (RL); and chest (precordial) electrodes, which can vary from one unit to six units (V_1–6). By convention, lead I measures the electrical activity from RA to LA, lead II measures of the electrical activity from RA to LL, lead III measures the electrical activity from LA to LL. The rule lead I + lead III = lead II makes it possible to derive a lead based on the other two. Leads I, II and III are known as Bipolar Limb. Unipolar leads measure the electrical activity from the Wilson’s central terminal (negative pole) to each of the chest electrodes (positive poles). For example, the Shimmer3 ECG is a four-lead ECG device wired with four extremity electrodes and one chest electrode, enabling the measurement of three bipolar and one unipolar lead.

For the sake of simplicity and to avoid verbosity and follow the terminology commonly used in the industry, we decided to represent an ECG device according to its ECG leads, classifying an ECG lead as a Sensor, since a sensor “detects and responds to events or changes in the physical environment” (SAREF). A lead can be either bipolar or unipolar, and an ECG device consists of at least one Bipolar Limb and one Unipolar lead. An ECG lead measures the Heart Electrical Activity property and makes measurements of ECG Sample Sequence. Figure 4 shows the core concepts of the SAREF4health ontology discussed here in an OntoUML model.

6.1.Responding competency questions

The first part of the SAREF4health validation was performed by specifying the ontology in RDF, using Menthor (Moreira et al., 2016) and Protégé. Instances of SAREF4health were created (as examples), and basic SPARQL queries were executed to answer each competency question listed in Table 2. As illustrated in some SAREF4health instances in Figure 4, the examples are based on the Shimmer3 ECG device, the device API (TinyOS), the mobile app (Android) as field gateway, and an IoT platform context broker playing the role of cloud gateway. Table 4 summarizes the results. Because of space limitation, we omitted textual properties (e.g., labels and comments). The SPARQL queries used are based on the following template:

All the competency questions listed in Table 2 could be answered, which is an indication for the completeness of SAREF4health and, therefore, its semantic validity. Our ontological commitments were driven by the definitions listed in Table 3. The ontology structure was assessed by analysing the SPARQL results (Table 4). For example, time series element (CQ05) is equivalent to sample sequence (UFO ECG) and Sampled Data (FHIR). The SPARQL results presented in Table 4 also show that the structure allows the representation of an ordered frequency-based sequence of float data, through the hasValues only xsd:float axiom (in CQ05), which allows the representation of multiple values and the serialisation as a compact array in JSON-LD.

SAREF4health quality was leveraged by the application of the best practices for ontology engineering adopted by our methodology. We argue that SAREF4health has a high quality because of the ontological foundations and practical matters, such as making SAREF4health as dereferenceable as possible and checking and reusing popular and common ontologies. A limitation of SAREF4health is that it still lacks a concept to represent the physician(s) administering an ECG recording session, which is common in other standards (e.g., DICOM). To overcome this issue, we suggest to add a class axiom to ECG Recording Session with the dc:creator, similarly to the approach taken with dc:author.

6.2.Semantic IoT early warning system prototype

In the second part of the validation, we connected a device (Shimmer3 ECG unit) to a field gateway (Android smartphone) to a cloud gateway (MS Azure IoT Hub). The goal of this validation is to show that SAREF4health has the adequate coverage for the envisioned INTER-IoT scenario, and also provides an adequate performance in terms of message verbosity and processing time.

The INTER-IoT-EWS prototype improves the manufacturer’s Android app (Shimmer Xamarin Capture) by translating the data received from the ECG device to SAREF4health and enables different rates of data exchange between device-mobile and mobile-cloud, which is a common network bandwidth optimisation requirement. This scenario allowed us to exploit the SAREF4health approach for time series measurements.

Fig. 5.

An IoT EWS to detect accident risks and accidents at the port of Valencia.

Figure 5 illustrates the solution architecture of INTER-IoT-EWS, having the MS Azure MyDriving application, which exemplifies the monitoring of vehicle trips, being able of tracking real-time location, speed and acceleration data through a mobile device connected to the cloud infrastructure (open source). The MyDriving mobile app behaves as a field gateway publishing data in the IoT Hub as JSON messages. Since the MyDriving app does not represent data as RDF, changes in the mobile app were required to turn the application to a MyDriving for Linked Data (MyDriving-LD) version. MyDriving-LD plays the role of the Semantic Field Gateway component of SEMIoTICS, providing logistics data, and thus requiring an ontology that represents transports, trips, location, acceleration and the goods transported.

MyDriving-LD had to be integrated with the Shimmer3 ECG device, using the ShimmerCapture solution as an example. For heath data, we applied the SAREF4health. Finally, ontology modules produced by TNO were used for logistics data2828: (1) Logistics Core Ontology (LogiCO) ontology, along with its extensions Transport (LogiTrans) and Logistics Services (LogiServ) ontologies.

The Shimmer ECG 3 device collects ECG data from the truck driver and sends the data to the MyDriving-LD application via Bluetooth at a high-frequency in real time. MyDriving-LD is deployed in a smartphone and plays the role of a field gateway, also managing the logistics data. A semantic wrapper is implemented in MyDriving-LD to allow the annotation of aggregated data with SAREF4health and the logistics ontologies. These annotated data are sent to the cloud as JSON-LD messages, i.e., published in the cloud gateway (Azure IoT Hub).

In the main configuration, the INTER-IoT-EWS Input Handler subscribes to receive all messages published in Azure IoT Hub, checking each message type (the domain: IoT, health or logistics) and certifying whether translations to harmonize the data in the SEMIoTICS core ontology are necessary. If this is true, the Input Handler requests the translations to the INTER-IoT semantic mediator (IPSM) (Ganzha et al., 2017a), forwarding the input message and the alignment required (pre-configured) for executing the translations. The IPSM is responsible for syntactically and semantically translating the data streams. After the semantic translations execution, the Input Handler is responsible for the syntactical translation to internal POCO classes that follow the SEMIoTICS core context model, which are used for both Context Data and Situation Identification managers. Once a situation is identified, the Situation Reaction component (see Figure 5) checks the workflows (business processes) to be triggered, which describe the different targets and their information requirements. The output handler formats the data according to the SEMIoTICS notification ontology (from OASIS EDXL-CAP) and forwards the messages with the appropriate information for each target.

The validation plan checked whether the requirements (see Section 4) were fulfilled. The functional validation activities included factory acceptance tests (FAT) and site acceptance tests (SAT) through a pilot in the port of Valencia, where accidents were simulated in accordance with the port emergency exercises. Both FAT and SAT assess whether the system works for the intended risks’ detection and warnings. The FAT was preformed to check completeness and requirements verification, simulating each of the situation types of each use case. Test hooks included data injection to increase the instantaneous acceleration data for the collision detection and to change the processed heart rate to below the bradycardia threshold and above the tachycardia threshold, as well as other more complex data patterns. An UI prototype was developed to support the execution of the tests.

The overall INTER-IoT-EWS performance analysis included the measurement of the total transaction time for the whole EWS workflow. We followed a similar validation method of the Semantic IoT EWS approach for scalability and resilience, testing upstream data acquisition and situation identification, measuring total transaction time from one component to another. The validation also considered the multi-brokering multi-cluster approach, showing that an industry-oriented cloud infrastructure, such as MS Azure, is able to scale the throughput threshold of 700 msg/sec/unit from the TRIDEC project (Middleton et al., 2013). In terms of efficiency of publishing semantically rich sensor data with a multi-broker approach, we conclude that this threshold (700 msg/sec/unit), considering a message size of 6 Kb (4.200 Kb/sec/unit), is easily reached with the Azure cloud infrastructure. The IoT Hub quotas and throttling supports the throughput of 6.000 msg/sec/unit (“device-to-cloud”), considering a message size of 4 Kb (24.000 Kb/sec/unit). Therefore, the cloud infrastructure can scale up to five times the broker throughput of the original approach (TRIDEC).