How The New AI Model For Rapid COVID-19 Screening Works?

With the current pandemic spreading like wildfire, the requirement for a faster diagnosis can not be more critical than now. As a matter of fact, the traditional real-time polymerase chain reaction testing (RT-PCR) using the nose and throat swab has not only been termed to have limited sensitivity but also time-consuming for operational reasons. Thus, to expedite the process of COVID-19 diagnosis, researchers from the University of Oxford developed two early-detection AI models leveraging the routine data collected from clinical reports.

In a recent paper, the Oxford researchers revealed the two AI models and highlighted its effectiveness in screening the virus in patients coming for checkups to the hospital — for an emergency checkup or for admitting in the hospital. To validate these real-time prediction models, researchers used primary clinical data, including lab tests of the patients, their vital signs and their blood reports.

Led by a team of doctors — including Dr Andrew Soltan, an NIHR Academic Clinical Fellow at the John Radcliffe Hospital, Professor David Clifton from Oxford’s Institute of Biomedical Engineering, and Professor David Eyre from the Oxford Big Data Institute — the research initiated with developing ML algorithms trained on COVID-19 data and pre-COVID-19 controls to identify the differences. The study has been aimed to determine the level of risk a patient can have to have COVID-19.

Also Read: This AI Model Can Predict If You Are A Job Hopper Or Not

Pre-Training Methodology For Research

With hospitals, currently using electronic healthcare records, according to the researchers, can be combined with machine learning techniques to develop an AI tool that can rapidly screen the COVID-19 virus in patients.

To initiate the model development, the researchers gather the electronic clinical data of all the patients coming to the emergency service as well as to get admitted at the Oxford University Hospitals from December 2017 to April 2020, which included around 170,510 sequential presentations. Each of the presentations included patients’ necessary information along with their blood tests, vital signs and other clinical information. 

Out of the data collected, the ones prior to December 2020 acted as a negative cohort for COVID-19, whereas, the ones presented between December to April 2020 were considered the positive cohort. The data collected is then investigated on five clinical variables — blood test, blood gas readings, changes in blood tests from pre-admission baseline, vital signs, and comorbidity index. 

Further, to attribute the missing data of the patients, researchers leveraged population mean, population median, and age-based data.

In order to real-time predict the COVID-19 presentations amid the patients admitted in the hospitals, the researchers applied linear logistic regression along with random forests, and extreme gradient boosted trees. These will model a non-linear relationship between input values and output values to separate the emergency department patients from the ones admitted in the hospital due to COVID-19 from pre-pandemic controls. And both the presentations of patients were predicted on separate models.

Also Read: How This Open-Source Spatial AI System Accelerates Computer Vision Functions

How Has The Model Been Trained?

According to the research paper, the model was trained on the data collected from December 2017 to April 2020, where the researchers “performed stepwise addition of clinical variables.” Also, the model was assessed on its performance using a stratified 10-fold cross-validation technique which separated the original data sample into an 80% training set and 20% testing set.

Schematic overview of model training, calibration & testing

To initiate the process, researchers first trained the model on the blood test variable and then gradually added up the other variables, also known as feature sets. Model thresholds were then measured to achieve maximum sensitivity for identifying the infected patients.

Alongside, each independent feature set, the blood tests of the patients, blood gas report, vital signs as well as the delta blood, were trained on Logistic Regression, Random Forest and XGBoost. Here it was highlighted that both the ensemble learning methods — Random Forest and XGBoost outperformed the logistic regression method due to the capability of detecting the non-linear effects of the variables. And out of the two, XGBoost turned out to be the most predictive with AUROCs of 0.904 (0.000) on blood tests and 0.823 on vital signs, for detecting COVID-19 infections.

After evaluating all the clinical variables and their training output, researchers decided to develop context-specific models using XGBoost classifier. This was trained to only use the regular clinical data to separately predict COVID-19 for emergency wards and admission wards with 77.4% sensitivities for both. One significant advantage to this model was the availability of the data, as it didn’t require any previous health data of the patient, becoming applicable to all.

The models, according to the researchers, proved to be extremely useful for predicting the patients at various stages of the pandemic – 92.3% and 92.5% accuracy for ED and Admissions models, respectively. Further, to simulate real-world performance at different levels of COVID-19, the researchers generated test datasets with varied cases of infected patients and assessed predictive values. The model was validated on all patients in the emergency and admitted to their hospital post-April 2020 data till 6th May 2020, and results were compared with the PCR test results.

Also Read: The Largest CAD Dataset Released With 15M Designs

Wrapping Up

Although the work of Oxford researchers can effectively speed up the COVID-19 screening process, the models come with a potential limitation of ethnic diversity in the data. And that’s why the researchers are now planning to integrate data from international centres to increase the model stability.

Having said that, until now, the early detection models were only used for radiological imaging; however, this pandemic has brought in the urgency to bring such techniques for accurate COVID-19 testing. Thus, this research was aimed at rapidly scaling the clinical need of AI to flatten the pandemic curve, without misclassifying the data based on ethnicity, age or gender biases.

Read the whole paper here.