Assessment of the disease severity in patients hospitalized for COVID-19 based on the National Early Warning Score (NEWS) using statistical and machine learning methods: An electronic health records database analysis

1.Introduction

The coronavirus disease 2019 (COVID-19) caused great concern in the health system. Many studies have been conducted to determine what might be associated with poorer outcomes and a more severe disease course. Healthcare institutions are in high demand of tools for quick evaluation of patient status severity or resource planning. However, assessments of the severity of the condition often vary, ranging from outcomes like in-hospital death [1, 2, 3] or need for intensive care [4] to specific health indicators like respiratory distress [5, 6, 7] or organ failure markers [8, 7]. Although there is an essential need to stratify patients according to their disease severity, the diversity in assessments makes it challenging to establish a standardized approach to measure and compare the severity of COVID-19 cases across different healthcare facilities and regions. Consequently, there is a demand for a comprehensive severity scoring system that incorporates various aspects of the disease course and provides a more holistic view of patients’ conditions. However, the question of which specific indicators best represent a patient’s overall condition during their hospital stay remains an object for discussion.

Often, especially when analyzing real-world healthcare data such as electronic health records, the ability to assess possible indicators is limited by the information recorded in databases during routine patient care. The most common tools to assess conditions used in practice are patient examination records with an evaluation of the patient’s vital signs, laboratory tests, treatment statuses such as the need for intensive therapy, and medical history of chronic diseases. Scientific data suggests that some tools useful for ongoing health monitoring and incorporated into routine practice may be appropriate for generally assessing patient’s condition.

The choice of scoring system depends on various factors, including the clinical context, patient population and there are many alternatives, such as MEWS [9], PEWS [10], SOFA [11], qSOFA [12], PRISM [13], APACHE II [14], NEWS [15] has gained significant recognition and widespread adoption in many healthcare settings due to standardization, extensive validation as well as integration into clinical pathways. The National Early Warning Score (NEWS) is often used to assess the condition of patients at the time of hospitalization [16] or to monitor their condition during hospitalization [17]. Although it is designed to assess immediate severe health events, usually limited to a 24-hour interval, it may be used to assess the overall complexity of patients’ conditions, for example, as an additional indicator to monitor the number of severe patients in the hospital.

This study aimed to use anonymized electronic health records from an existing healthcare facility to assess the potential of routinely performed NEWS measurement (alone or combined with other clinical data) to assess the patient’s health status during hospitalization period.

2.Materials and method

2.2NEWS score and its measurements during hospitalization

As a clinical tool used to assess the severity of a patient’s illness and determine their risk of deterioration, the NEWS is based on a set of physiological parameters, each assigned a score depending on their abnormality. These parameters include respiration rate, oxygen saturation, systolic blood pressure, heart rate, body temperature, level of consciousness. If a parameter falls within the normative range defined in the NEWS calculation chart [15, Fig. 1], it receives a score of 0. As the parameter deviates from the normative range, the score increases accordingly. The parameters’ scores are then added up to give the total NEWS score (0–20 points). The NEWS data utilized in the analysis were derived from the patient’s initial recorded NEWS values, representing the median of all documented NEWS scores available for each participant.

Figure 1.

Schematic representation of the models’ development workflow.

3.Results

Patients with positive outcome significantly differed from those with negative with respect to majority of characteristics (see Table 1; insignificant differences were observed only with respect to gender and two lab measurements).

Models’ quality was evaluated by making use of several common scoring metrics (for a full list see Subsection 2.7 and for complete numerical results are presented in Table 2). However, for ranking, we relied on F1 score and AUC. Concerning F1, the top three lines were as follows: stepwise logistic regression with L1 regularization (F1 = 0.859), SVM with the radial kernel (F1 = 0.855), and stepwise logistic regression model without any regularization (F1 = 0.850). This last model was the best to AUC (0.9835), yet its advantage over L1 regularized (logistic regression) version was minimal (AUC = 0.9827). Finally, SVM with radial kernel ranked third (AUC = 0.9694). Since the difference between pure machine learning models and unregularized stepwise logistic regression (USLR) was minimal, we have further focused on USLR, intending to extract information regarding significant predictors (recall that USLR is, in fact, a statistical model and, therefore, provides a model-based ability to interpret the significance of regressors). In USLR, the following variables were significant: age, ICU need, first NEWS, Median NEWS, CPB, BNP, ALT, AST, lactate, creatinine, urea and an indicator of at least one chronic disease (see Table 3 for details).

Since our goal was to contrast the single NEWS and NEWS’s predictive ability with variables, we have also built a logistic regression model based only on Median NEWS. The latter model had F1 = 0.719 and AUC = 0.942. Figure 2 provides a graphical summary of our analysis in terms of ROC curves.

Figure 2.

ROC curves of all models and best models: (A) logistic regression models, (B) machine learning models, (C) best models and single NEWS model.

4.Discussion

Based on our analysis, we can conclude that single Median NEWS has a high predictive ability when predicting in-hospital mortality. The results coincide with those obtained by other researchers who considered NEWS and NEWS2 for the same purpose [19, 20, 21, 22, 23, 24]. However, combined with other routinely taken measurements, the predictive value can increase even more. This is again in agreement with findings of others [19].

In our case, advanced Machine Learning models did not surpass commonly used logistic regression (LR). Several studies also report the same result [26, 27]. Since LR has an advantage of model-based interpretation of significance of variables employed in the model, it can be considered as a primary option for integrating NEWS into routine usage in a form of predictive models investigated by us. For this purpose, we are going to develop an enhanced version of the model suitable for predicting with incomplete data and integrate it into the hospital information system in a form of a dynamic Microsoft Power BI Report Server based webpage. In addition to the previously mentioned problem of incomplete data, there are other challenges to address within a frame of this task. The shortlist is as follows:

Summing up, our pilot study reveals several new directions of NEWS usage and poses new challenges. To our best knowledge, there are not many studies devoted to the analysis of NEWS as a measurement for making resumptive conclusions. Therefore, our input seems reasonable.

We hope that soon we’ll be able to make good use of NEWS in combination with other routine measurements both for monitoring purposes and for summarizing the whole in-hospital period as in the present study. This way, it will be possible to expand the present use of this score by practitioners and decision-makers.

5.Conclusions

Our model demonstrated incremental value for COVID-19 patient’s status evaluation. The National Early Warning Score (NEWS) is a valuable tool to predict in-patient mortality and its performance can be enhanced more by combining it with other clinical data, such as age, ICU need, at least one chronic disease, CPB, BNP, ALT, AST, lactate, creatinine and urea. Machine learning techniques showed a reasonably high accuracy of models. Nevertheless, logistic regression models remain compelling, especially when using the regularization approach. Incorporating the NEWS into hospitals may be a good option for monitoring purposes. However, processing incomplete data in a real-world setting is essential for optimal results and needs more attention in future studies.

Ethics statement

The research was authorized by the Vilnius Regional Biomedical Research Ethics Committee (permission no. 2022/2-1411-882). Personal patient consent was not required since anonymized data was used.

Data availability statement

The data presented in this study are available from the authors upon reasonable request, following institutional approval.

Author contributions

Conceptualization: V.L., R.P., G.T., G.B. and A.J.; Methodology: V.L., R.P., V.S., G.T., G.B., E.K.; Software: V.L.; Validation: G.T.; Formal analysis: V.L.; Visualization: V.L., R.P., G.T. and J.T.; Interpretation of the results: V.L., R.P., P.N., V.S.; Data curation: V.L., R.P. and G.T.; Supervision: J.T., G.B., E.K. and A.J.; Writing original draft preparation: V.L., R.P., V.S., P.N. and G.T. All authors have read and agreed to the published version of the manuscript.