A Computational Approach in the Diagnostic Process of COVID-19: The Missing Link between the Laboratory and Emergency Department

Background : The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is responsible for the COVID-19 pandemic and so it is crucial the right evaluation of viral infection. According to the Centers for Disease Control and Prevention (CDC), the Real-Time Reverse Transcription PCR (RT-PCR) in respiratory samples is the gold standard for confirming the disease. However, it has practical limitations as time-consuming procedures and a high rate of false-negative results. We aim to assess the accuracy of COVID-19 classifiers based on Arificial Intelligence (AI) and statistical classification methods adapted on blood tests and other information routinely collected at the Emergency Departments (EDs). Methods : Patients admitted to the ED of Careggi Hospital from April 7th–30th 2020 with pre-specified features of suspected COVID-19 were enrolled. Physicians prospectively dichotomized them as COVID-19 likely/unlikely case, based on clinical features and bedside imaging support. Considering the limits of each method to identify a case of COVID-19, further evaluation was performed after an independent clinical review of 30-day follow-up data. Using this as a gold standard, several classifiers were implemented: Logistic Regression (LR), Quadratic Discriminant Analysis (QDA), Random Forest (RF), Support Vector Machine (SVM), Neural Networks (NN), K-nearest neighbor (K-NN), Naive Bayes (NB). Results : Most of the classifiers show a ROC > 0.80 on both internal and external validation samples but the best results are obtained applying RF, LR and NN. The performance from the external validation sustains the proof of concept to use such mathematical models fast, robust and efficient for a first identification of COVID-19 positive patients. These tools may constitute both a bedside support while waiting for RT-PCR results, and a tool to point to a deeper investigation, by identifying which patients are more likely to develop into positive cases within 7 days. Conclusions : Considering the obtained results and with a rapidly changing virus, we believe that data processing automated procedures may provide a valid support to the physicians facing the decision to classify a patient as a COVID-19 case or not.


Introduction
The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is responsible for the COVID-19 pandemic.Since its first report in December 2019 [1], despite great efforts made in almost every country worldwide, this disease continues to spread globally.
According to the Centre for Disease Control and Prevention indications, the upper respiratory samples, and in particular the nasopharyngeal specimen, should be collected for RT-PCR based testing of COVID-19 [2].
Unfortunately, RT-PCR has several practical limitations due to time-consuming procedures and to problems related to the detection rate of viral nucleic acid closely related to the course of viral infection.The optimal sampling time is uncertain and subsequently the period of the high viral load can be missed, resulting in a high rate of falsenegatives [3], while there is a growing interest in the role of biomarkers in the screening and especially the early detection of SARS-CoV-2 infection at emergency departments (EDs) [4,5].
The aim of our study was to assess, through classification tools based on machine learning (ML) and standard statistical models, the predictive capacity to classify patients as COVID-19 positive cases from the first wave (April 2020), starting from diagnosis based on blood tests and other information routinely collected at the emergency department.With a rapidly changing virus, we believe that data processing automated procedures may provide a valid support to the physicians facing the decision to classify a patient as a COVID-19 positive or not.We tested different classifiers to find the best way to combine ML and statistical methods in the COVID-19 context, in order to improve their effectiveness while preserving the results' interpretability.
Given the above recalled limitations of RT-PCR test and as far as the models were developed in early 2020, when the first wave of COVID-19 occurred and the timing to swab was still not well known, the benchmark considered to train the classifiers is the so called physician's gestalt, which stems from a revision of clinical, laboratory (including RT-PCR), imaging parameters and a clinical review of 30-day follow-up data.This has been shown to play a key role in identification of patient infection in the first wave [6].
In addition, the classifiers were tested on data from patients from the second wave (October 2020) using, in this case, as a benchmark, the positivity to the specific RT-PCR within one week from the access to the ED (not available in the first dataset).This is in line with the growing body of evidence suggesting that missed positive patients may disclosure within 7 days from the admission to hospital [7,8].

Dataset and Analytical Systems
This study was conducted after approval by the local ethical committee and informed consent was submitted and signed to the involved patients.
971 consecutive patients at their admission to the ED of the hospital AOU-Careggi were enrolled starting from April, 7th to April 30th 2020.Inclusion criteria were: age ≥18 years and first ED evaluation during the study period.Exclusion criteria were: known diagnosis of COVID-19, patient's loss at follow-up after ED discharge with a negative RT-PCR test result and patient's refusal to participate in the study.
Of these, 118 were later classified as cases according to the physician's gestalt as described in Nazerian et al. [6], while the COVID-19 negative were diagnosed with all the other diagnosis; no noticeable repetition found.
Serum, plasma and blood from peripheral venous sampling were evaluated upon admission to the ED.Hematochemical parameters were evaluated on specific analytical platforms.In particular: total blood cells count was performed on Sysmex XN analyzer (Dasit, Milan, Italy), while coagulation and serum biomarkers were evaluated after centrifugation, plasma samples were tested on ACL-TOP analyzer (Werfen, Barcelona, Spain) and sera on Cobas 8000 analyzer (Roche, Barsel, Switzerland).
The laboratory variables considered for our analysis are listed in Table 1: We also included the following variables (Table 2) not strictly laboratory related, but easily assessable at the first emergency: Table 1.Laboratory parameters measured upon admission to the ED.
Dyspnea (yes/no) Notably, both fever and body temperature were considered: in particular, fever was referred to at home selfassessment of body temperature higher than 37.4 • C and considered as a qualitative covariate, while Body Temperature was measured upon the admission to the ED and considered as quantitative.
Given the characteristics of the first wave, mainly affecting male and geriatric people, we deliberately excluded from our sets of covariates age and sex.The inclusion of these covariates would have improved the internal validation based on the first wave but would have had a worsening effect of the classification power in subsequent waves.
We decided to use a testing population patient from the second wave, in order to overcome overfitting matters as reported in literature [9].Patients were selected by consecutive admission to the ED from 1st to 24th October 2020.In particular, we used the data from the first 30 admitted patients who resulted to have a positive molecular test for COVID-19 within one week and 30 patients with any other diagnosis (no specific repetition among diagnoses were noticed).

Statistical Data Analysis
Recent developments in ML concern automated procedures to classify units.We here compare seven different technological approaches that, according to literature, provide optimal results [10].
The analyses were performed using the R version 3.6.3(Vienna, Austria), via the package CARET (Classification and REgression Training).
We used a wrapper method for the feature selection, combined with cross-validation steps to improve the selection [11,12].The proposed recursive feature elimination (RFE) (backward selection algorithm) was combined with a RF model, and 10-folds cross-validation repeated for 5 times [11,12].The set of features for each algorithm was determined through hyperparameter optimization (grid search).To tackle the unbalancedness, the Synthetic Minority Oversampling Technique (SMOTE) computational method to artificially generate units from the minority class via the K-nearest neighbor (K-NN) method [13] was used.

Results
The pattern of missingness of the laboratory variables is displayed in Fig. 1.The variables AST, Dim and TnT show more than 75% of missing values and therefore were removed from further analysis.Also other variables (namely Fib, PCT and IL-6) have a rather high percentage of missingness.However, given their importance, we decided to keep them for further analysis.
Prior the application of various ML techniques, for each of the laboratory variable, we investigated if some transformations (such as log transform, quadratic power, etc.) may lead to an improvement of their discriminatory power, i.e., their capacity to separate positive from negative patients.This analysis led us to conclude that the variables, Ly, LDH, K, Bil, Crea, PCT, INR, IL-6, ALT, may be better imputed in logarithmic form (after adding 0.01 when necessary).
Additional preliminary analyses were performed in order to understand the influence of each covariate on the conditional log odds ratio of a patient to be COVID-19 positive.These concern a semi-parametric Logistic Regression (LR) against each variable and a parametric LR against each variable binned into quartiles.The former uses smoothing techniques based on cubic splines (i.e., third order polynomials constructed to achieve smoothness of the interpolating function) [14], while the latter is performed by grouping patients into quartiles, and then fitting a parametric logistic model against the binned variable considered as a factor.As an explicative and representative example, in Fig. 2 the resulting plot for the sO2 variable is presented.Both graphs show that for this covariate, after the level of 85 there is a dramatic decrease of the log odds that the patient is a case.
In order to apply the ML and statistical methods, the dataset was randomly divided in training (80%) and validation set.Therefore, the training set is formed by 778 patients, of which 95 are cases.Furthermore, data coming from the early-second wave (Oct 01, 2020-Oct 24, 2020) are also used to verify the accuracy of the classifiers.We will refer to them as internal (first wave)/external (second wave) validation sample.Further preliminary data analysis in the training datasets were performed adapting the method of Perez-Riverol et al. [15].In detail, after using K-NN (k = 5) to impute missing values and a cupping procedure for outliers, the Ne variable is removed as it exhibits a large correlation (cutoff 0.7) with WBC and presents higher rate of missing values in the original dataset.A near zero variance analysis did not point to further removal.
On the training set, we then used a wrapper method for the feature selection and cross-validation steps to improve the selection [5,15].The proposed recursive feature elimi- Observed log odds obtained after binning patients into quartiles plotted against the central point of the interval between consecutive quartiles (in blue the observed log odds for patients with missing value).The preliminary analysis suggests a possible addition of a quadratic term of the sO2 variable, which, however, does not result in a significant effect when considered jointly with the other covariates.nation (RFE) (backward selection algorithm) was combined with a RF model, and 10-fold cross-validation repeated five times [10,11].This selection algorithm ranks the variables iteratively according to their importance (determined using RF) and, at each stage, the least important predictors were eliminated.The set of hyperparameters for each algorithm was determined through hyperparameter optimization (grid search).To tackle the unbalanceness, the Synthetic Minority Oversampling Technique (SMOTE) computational method to artificially generate units from the minority class via the K-nearest neighbor (K-NN) method [13] was used.
The repeated use of RFE indicated 16 covariates as relevant: Fever, Cough, LDH(log), WBC, CRP, Plt, Ly(log), Dyspnea, Fib, Bil(log), SO2, Na, Hb, PCT(log), Body Temperature, IL-6(log).On these 16 variables, several classifiers have been implemented both parametric (namely LR and QDA) and non-parametric (namely RF, SVM, NN, K-NN, NB).In implementing the above classifiers, a Spatial sign transform on the data has been applied, as it is known that this transform may achieve a better discriminatory power [16], a finding which is confirmed in our analysis.The imputation method was only used on the training set in order to preserve the validity of the cross-validation.
Almost all classifiers exhibit a high value of the area under the ROC curve (AUC), but the best results are obtained when classifiers are implemented on the log transformed data with the rebalancing SMOTE technique.In this case, with the only exception of SVM and KNN, all methods applied to the Internal Validation dataset present a ROC above 0.80 (see Table 3).The external validation also exhibits a high level of accuracy of precision (see Table 4), when applied on the log transformed data with the classifiers estimated with the SMOTE technique.In addition, all classifiers exhibit a good level of Specificity and Sensitiv-ity in all the tested scenarios, confirming the efficacy of the proposed approaches.Notice that, depending on the aim of the classification, Sensitivity could be increased but at the Specificity expense.We documented that are the classifiers based on RF, NN and LR are particularly successful.Since the latter has a parametric form that naturally lends into a scientific interpretation, next section focuses on the estimated parameters obtained via implementation of such model.

Focus on the Logistic Model
In Table 5 we reported the estimates of the parameters, together with their standard errors and p-values (in order to preserve the interpretation, all variables are here in the original scale, with the exception of the log transformation on the LDH, Ly, Bil, PCT, IL-6).The point and 95% confidence interval estimates of the odds ratios are also reported.All non-significant variables on SMOTE data appeared to improve the precision of the classifier in the external validation, and therefore were kept in the classifier (the only exception is Body Temperature).All variables acted in the expected direction, with the exception of Bil and PCT that exhibited a significant negative effect.The log odds to be a case of a patient with Dyspnea was 0.364 higher (that corresponded to an increase of 1.1370 on the odds ratio scale) than a patient without, when both had the same value of the other covariates.Similarly, at any level of the other covariates, a one-point increase of sO2 decreased the log odds to be a case of 0.539 (that corresponded to a decrease of 0.583 on the odds ratio scale).

Discussion
In the last few years, several research groups are developing ML methods to support daily clinical practice and the worldwide pandemic enhanced the efforts in that side [5].Our work applied ML methodology and classical statistical classification models to data coming from routine blood exams, which are commonly requested at the admission to the ED.Such exams are ready in a short time and much cheaper than molecular test or radiological examinations.
Furthermore, despite the fact that the RT-PCR test is still the gold standard for conclusively diagnosing COVID-19 infection, there are some concerns about its clinical performance, which is affected by a number of difficult-tomeasure factors like low levels of shedding during incubation and, despite its known high specificity, its sensitivity is still debatable [17].As a matter of fact, among initially negative patients subjected to repeat SARS-CoV-2 RT-PCR testing, newly positive results within 7 days at the time of initial presentation may occur [7].
Notice that, while during the first wave physicians were facing an unknown disease, in the second wave several limitations (as the high rate of false RT-PCR negatives) were already well documented.As a result, whereas the 30-day review was absolutely necessary for the first wave, it was not required for the second wave because new molecular approaches had been created.This validates our decision to employ the second molecular test when applied to the second wave data and our use of the physician's gestalt when training the model on the first wave data.Since the two COVID-19 waves were markedly different in terms of demographic characteristics, we deliberately excluded from the training dataset age and sex.Keeping them into the models would have had a worsening effect in the external validation sample.
LR can be seen both as a good classifier and a tool to give scientific insights on the subject matter.A thorough analysis of the calculated parameters reveals that, all 16 factors affect a patient's likelihood of being a case in the predicted way, with the exception of Bil and PCT, which have a considerable adverse effect.More investigations are required on this aspect.A possible explanation is that the two variables do not influence the probability of a patient to be a COVID-19 case.If so, the unfavorable connection may be an artifact of the sampling population's selection procedure.In fact, patients who present at the ED with high values of these variables are influenced by diseases other than COVID-19, assuming the supposition of no effect is accurate.Since it is rather unlikely that they are affected by the two diseases at the same time, this fact may induce the negative association.
Limitations of our study come from the combined effect of data unbalancedness and missingness.This surely impacts mostly on the sensitivity of the classifiers, but our results are however in agreement with previous and recent studies [5,18].Further limitation may come from the fact that, at present, blood tests are not needed, exception done for patients with severe COVID-19 symptoms or with complications due to co-morbidities that, however, may affect blood test results.Recently, other tools considering clinical and epidemiological features are developed and well described in literature to build risk score to predict SARS-CoV-2 infection [19].We claim that training of the model could be greatly improved if all the covariates that resulted into relevant according to this and other studies, are measured for all patients.By doing so, the problem of unbalancedness can be addressed in a more robust way via the SMOTE rebalancing approach, and the ML and statistical methods here described would then constitute a valid instrument for rapid assessment of a potential COVID-19 positive patient.
In case of another pandemic, new data should be collected in order to train the classifiers and this may constitute another limitation.
Anyways, our study is innovative in that we tested for positivity within a week, in line with the majority of recently published research studies [7,8].We want to underline that proposed algorithms do not claim to be alternative to the gold standard RT-PCR, but rather to provide an additional and impacting complementary information.This can be used either when the RT-PCR is missing or to identify, among patients with a negative RT-PCR SARS-CoV-2, which ones are more likely to result into cases within the 7days window, although more recent studies demonstrated that only patients with a very high clinical probability and an initially negative result of the RT-PCR justify the need for retesting [20].

Conclusions
In this paper we demonstrated that ML and classical statistical methods may be applied to common blood tests upon admittance to the emergency departments when there is the clinical suspect of COVID-19, given their affordability and fast effectiveness.
The focus on the logistic model allows to better understand data and their trend, while ML models are kind of black boxes where a complete intuition of the data transformation remains unknown.
We underline that the use of automated processes to classify cases may significantly aid clinicians in dealing with the constantly evolving COVID-19 virus.In this way, our paper serves more as a manual on how to apply these techniques and what benefits and drawbacks they might have.This will support the idea that well trained and properly tested, routinely updated, automated classifiers may help clinicians' decision if implemented in the Laboratory Information System and the outcome of the classifiers made available to the ED.

Fig. 2 .
Fig. 2.An example of the preliminary analyses performed on all covariates.Here the effect sO2 on the log odds to be a COVID-19 patient is investigated via: (a) semi-parametric logistic model against the covariate (confidence intervals in grey).(b)