Andrew Ng recently participated in a virtual Q&A session hosted by DeepLearning.ai and Stanford HAI on AI in healthcare. When asked about the challenges of deploying AI systems in an actual clinical setting, Andrew said: “It turns out that just because someone publishes a paper showing AI performs at a level similar or maybe superior to a certified clinician, there remains a lot of work to take into production. All AI, and not just in healthcare, have a proof-of-concept to the production gap.”
Ng further said the entire cycle of the machine learning model goes beyond just modelling; it involves finding the right data, deployment, monitoring, and feeding data back to the model. “It goes beyond doing well on the test set, which fortunately or unfortunately is what we in machine learning are great at.”
Ng pointed at a persistent problem in the real-time adoption of AI in healthcare. Though AI has the potential to transform health and life sciences spaces, amplify R&D capabilities, and improve clinical decision making, it still has a long way to go.
AI in healthcare
AI in healthcare has the potential to solve challenges such as:
- Reduce unwarranted variation in clinical practice
- Improve efficiency and prevent avoidable errors
- Enable better care through novel tools that support patients
- Assist patients in managing their health
- Extracting novel insights from data
Studies show AI can match or best humans in many healthcare tasks. Today, we have algorithms outperforming radiologists in spotting malignant tumours and guiding researchers in constructing cohorts for costly clinical trials.
The significant challenges standing in the way of AI implementation in healthcare are:
Lack of high-quality reporting: Most studies are published directly on the
preprint servers and not submitted to peer-reviewed journals. Peer review is important for building trust to drive wider adoption of AI in the healthcare industry. High-quality reporting of machine learning studies is paramount to understand the risk of bias and the usefulness of prediction models. Studies point out there are only a handful of randomised controlled trials of AI systems to date.
Lack of data: Significant gaps exist in available data for different age groups, genders, and communities. Lack of data translates to lesser columns, leading to inaccurate diagnosis and bias. The data problem in healthcare is three-fold:
· Patient data is often challenging to access. Most data lie within charts, electronic records, and other sources. It requires a domain expert to convert all this data into a form that healthcare professionals can utilise. Additionally, several countries have strict restrictions on the use of patient data to preserve privacy; this too hampers data collection and access.
· As mentioned, most of this data makes little sense in its raw form. It is an expensive and time-consuming task to process the unstructured data.
· Often medical professionals lack the necessary insights to make an accurate diagnosis.
Inapplicability: In 2016, Google researchers developed a deep learning algorithm for interpreting Diabetic Retinopathy (DR) signs, a leading cause of blindness among diabetic patients. One of the team’s major challenges while developing these algorithms was the model’s non-applicability in real-world applications, even though they worked fine in controlled lab settings. Even smaller factors like lighting inside a room played a major role. AI algorithms can suffer from such inapplicability, leading to the brittleness of the model and even potential bias.
Generalisation: AI systems are yet to achieve reliable generability. A model with blind spots may result in poor performance. Generalisation in AI is a challenge, more so in healthcare applications, due to technical differences between the equipment and coding definitions and variations in local clinical and administrative practices.
MLOps: A possible solution
MLOps offer a set of best practices aimed at ML lifecycle automation that brings together the system development and operations aspects. Placed at the intersection of DevOps, machine learning, and data engineering, MLOps can bypass the bottlenecks that exist in the deployment process. Researchers have said MLOps can work well in critical industries such as healthcare and finance.
