Why Doctors Should Be Cautious While Using Deep Learning Applications In Clinical Settings


Machine learning is applied in various business areas, right from oil extraction to building self-monitoring optical systems. As the ML applications progressed, there has evolved a parallel concept of investigating loopholes, called adversarial attacks. If these attacks are not identified and mitigated quickly, it could leave the applications vulnerable to attackers. This is dangerous in critical business sectors such as healthcare and banking, and more so in the field of medicine, where the attacks may drastically affect human lives.

This article focuses on attacks in medical deep learning systems and looks into the adverse impacts it may have. We are considering a research study conducted by scholars at Harvard Medical School, where they explained practical use cases of adversarial attacks in medical imaging through deep learning, and showcases how its implementation can be deleterious on medical diagnosis.


Sign up for your weekly dose of what's up in emerging technology.

The Progeny Of Adversarial Examples

Deep learning (DL) is a practical application of ML algorithms through neural networks. Off late, deep learning has found significant improvement with respect to its implementation in healthcare environment — especially medical imaging in radiology. It delivers quicker and accurate results compared to manual diagnosis by physicians. Deep learning is also being used to enhance  operating efficiency.

Although the growth has been positive, this has led to researchers seeking out vulnerabilities in deep learning systems called ‘adversarial examples’. This itself has paved way to a separate path for research and has gained quick attention among ML enthusiasts. First discovered by Ian Goodfellow and others at Google, adversarial examples leave ML loopholes out in the open. This has inspired researchers to come up with ways to trick DL neural networks. For instance, methods such as Deepfool manipulate image-related DL systems to misclassify the images in large datasets. This could be harrowing and disastrous if applied in medical imaging.

Example of Adversarial attack: Picture showing misclassification of cancer skin cells due to perturbation (Image courtesy: Samuel Finlayson and Isaac Kohane)

Roots Of Vulnerability In Adversarial Attacks

In the study, the scholars first discuss the impact of fraud in the ever-growing healthcare sector in the United States. They emphasise that fraud is prevalent due to the size of the ecosystem and committing a fraud is possible at all organisational levels. Moreover, the data associated with healthcare is transforming electronically with popular bookkeeping methods on the rise. This is affecting the economy drastically, since every data related to healthcare is critical. Manipulating health-related data, even on a small scale and on a continuous basis, may be at the cost of many lives.

The scholars provide an outline of seven factors that expose DL systems’ vulnerabilities significantly. They are listed as is from the study, below:

  1. Ground truth is often ambiguous”: This means physicians/radiologists often disagree with image classification in medical imaging since it is relatively new and has spurred controversies even though it is found beneficial. Attackers take advantage of this fact since medical experts already have a negative mindset on DL imaging.
  2. Medical imaging is highly standardised”: The stringent standards followed in medical imaging may limit DL techniques and this leads to systems becoming more susceptible to attacks.
  3. Commodity network architectures are used often”: When it comes to using computer vision in medical imaging, all methods by researchers follow a similar approach. This means, the AI models in imaging have same network architecture. This makes it even more transparent for attacks.
  4. Medical data interchange is limited and balkanised”: The mechanisms for sharing medical data is limited and only few companies or vendors provide data interchange services among different healthcare organisations.
  5. Hospital infrastructure is very difficult to update”: It is difficult for large health organisations to upgrade their medical portfolio since software systems that cater to the entire organisation are complex and are tough to handle when it comes to revisions, updates and latest machinery.
  6. Medicine contains a mix of technical and non-technical workers”: Since the medical sector consists of employees and staff from various disciplines, the level of knowledge in ML varies drastically. For instance, an operator who works a medical imaging device might find it difficult to distinguish adversarial attacks, unlike a ML engineer who would analyse corrupt components in the data very quickly.
  7. There are many potential adversaries”: In statistics, it is said that uncertainty is a given for an experiment. Similarly, in this case, the avenue for unknown, newer adversarial attacks is uncertain given that along with positive developments, the bad side of it might also show a new face.

These factors highly determine how an attack creeps into the DL system. It is advised that any healthcare organisation following and implementing ML or DL should strongly keep these factors in mind.


DL applications are always enticing to solve real-world problems since it achieves operational efficiency as well as help reduce costs. With sectors such as healthcare and banking slowly seeing a rise, the feasibility of DL applications with respect to data security as well as data genuineness can thus be checked before implementation.

More Great AIM Stories

Abhishek Sharma
I research and cover latest happenings in data science. My fervent interests are in latest technology and humor/comedy (an odd combination!). When I'm not busy reading on these subjects, you'll find me watching movies or playing badminton.

Our Upcoming Events

Masterclass, Virtual
How to achieve real-time AI inference on your CPU
7th Jul

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, Virtual
Deep Learning DevCon 2022
29th Oct

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM