There have been well-studied epidemiology models that can help predict COVID-19 cases and deaths to help with these challenges. This pandemic has generated an unprecedented amount of real-time publicly available data, making it possible to use more advanced machine learning techniques to improve results.
In line with the same, Google AI proposed a framework designed to simulate the effect of certain policy changes on COVID-19 deaths and cases, such as school closings or a state-of-emergency at a US-state, US-county, and Japan-prefecture level, using only publicly-available data.
“We conducted a 2-month prospective assessment of our public forecasts, during which our US model tied or outperformed all other 33 models on COVID-19 Forecast Hub. We also released a fairness analysis of the performance on protected sub-groups in the US and Japan,” said the company in the blog.
The researchers proposed a few extensions to the Susceptible-Exposed-Infectious-Removed (SEIR) type compartmental model in this work. For example, susceptible people becoming exposed causes the susceptible compartment to decrease and the exposed compartment to increase, with a rate that depends on disease spreading characteristics. Observed data for COVID-19 associated outcomes, such as confirmed cases, hospitalisations and deaths, are used to train compartmental models.
In addition to quantitative analyses to measure the performance of their models, researchers conducted a structured survey in the US and Japan to understand how organisations were using our model forecasts. In total, seven organisations responded with the following results on the applicability of the model, describing count for real-life use cases.
- Organisation type: Academia (3), Government (2), Private industry (2)
- Main user job role: Analyst/Scientist (3), Healthcare professional (1), Statistician (2), Managerial (1)
- Location: USA (4), Japan (3)
- Predictions used: Confirmed cases (7), Death (4), Hospitalizations (4), ICU (3), Ventilator (2), Infected (2)
- Model use case: Resource allocation (2), Business planning (2), Scenario planning (1), General understanding of COVID spread (1), Confirm existing forecasts (1)
- Frequency of use: Daily (1), Weekly (1), Monthly (1)
- Was the model helpful?: Yes (7)
To share a few examples, in the US, the Harvard Global Health Institute and Brown School of Public Health used the forecasts to help create COVID-19 testing targets that were used by the media to help inform the public. The US Department of Defense used the forecasts to help determine where to allocate resources and to help take specific events into account. In Japan, the model was used to make business decisions. One large, multi-prefecture company with stores in more than 20 prefectures used the forecasts to better plan their sales forecasting and to adjust store hours.
However, the explained approach also has a few limitations:
- First, it is limited by available data, and we are only able to release daily forecasts as long as there is reliable high-quality public data. For instance, public transportation usage could be very useful, but that information is not publicly available.
- Second, there are limitations due to the model capacity of compartmental models as they cannot model very complex dynamics of COVID-19 disease propagation.
- Third, the distribution of case counts and deaths are very different between the US and Japan. For example, most of Japan’s COVID-19 cases and deaths have been concentrated in a few of its 47 prefectures, with the others experiencing low values.
This means that the per-prefecture models, trained to perform well across all Japanese prefectures, often have to strike a delicate balance between avoiding overfitting to noise while getting supervision from these relatively COVID-19-free prefectures.
“We have updated our models to take into account large changes in disease dynamics, such as the increasing number of vaccinations. We are also expanding to new engagements with city governments, hospitals, and private organisations. We hope that our public releases continue to help public and policy-makers address the challenges of the ongoing pandemic, and we hope that our method will be useful to epidemiologists and public health officials in this and future health crises,” their post concluded.
