Active Hackathon

Google Introduces New Machine Learning Tool To Boost Digital Pathology

Machine learning (ML) is now capable of assisting medical professionals in diagnosing critical cases like diabetic eye disease and metastatic breast cancer.


Sign up for your weekly dose of what's up in emerging technology.

No matter how improved the algorithms are, they will be useless unless a doctor finds the data sensible. So how doctors interact with information can be crucial determinants in the utility of ML technology.

Traditionally pathologists take reference of their colleagues and other sources like textbooks to examine the problem at hand. It can be an image of a cancer scan or of that of blunt trauma. Modern-day technology enables the computers to skim through thousands of images and retrieve information which is the closest to the query. For example, the reverse image search option in Google images. 

Now, there is widespread adoption of ‘digital pathology’. The clinicians can now examine the data received as images on a computer. And, can draw insights from it making the whole process of diagnosis relatively easy.

On these lines, a group of researchers at Google artificial intelligence department have teamed up to introduce machine learning tools and methods to propel the adoption of digital pathology:

  • Similar image search for histopathology – SMILY
  • Human-centered tools for coping with imperfect algorithms during medical decision-making 

Similar Medical Images Like Yours (SMILY)

The first step in developing SMILY was to apply a deep learning model, trained using 5 billion natural, non-pathology images (for example, dogs, trees, man-made objects, etc.) During the training, the network learned to distinguish between similar and dissimilar images by comparing the embeddings. 

Embedding is a summarization of the numerical vector which is a compression of all the images fed into the deep learning models during training.

SMILY allows a user to select a region of interest, and obtain visually-similar matches. As can be seen in the example below where selecting a small region in a slide will enable SMILY to efficiently search a database of billions of cropped images in a few seconds. 

Because pathology images can be viewed at different magnifications (zoom levels), SMILY automatically searches images at the same magnification as the input image. Though these machine learning tools for similar image retrieval drastically reduce the time taken through traditional methods, they fall short on understanding the intent of the user or searcher.

A doctor can look at the same report for different things and it varies from case to case. To make SMILY more interactive, the researchers refined the tool by enabling end-users to express what similarity means on-the-fly: 

  1. Refine-by-region allows pathologists to crop a region of interest within the image, limiting the search to just that region 
  2. Refine-by-example gives users the ability to pick a subset of the search results and retrieve more results like those
  3. Refine-by-concept sliders can be used to specify that more or less of a clinical concept be present in the search results (e.g., fused glands)

The above concepts need not be built into a machine learning model. Instead, this method allows end-users to create new concepts post-hoc, customising the search algorithm towards concepts they find important for each specific use case.

via Google AI blog

The above illustration depicts how interactive the tool is when augmented with refinement.

SMILY: A Clinical Approach

The study conducted by the researchers was published in Nature journal and the results show that the pathologists have found this new tool useful in decision making like keeping a track of the likelihood of a hypothesis.

Tools such as these make the doctors more inclusive in the digital process and eliminate the passivity that the existing methods bring with them. SMILY strikes the right balance by equipping the doctors with domain expertise while allowing them to customise the tool for their own benefit.

Know more about the tool.

More Great AIM Stories

Ram Sagar
I have a master's degree in Robotics and I write about machine learning advancements.

Our Upcoming Events

Conference, Virtual
Genpact Analytics Career Day
3rd Sep

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM
How Data Science Can Help Overcome The Global Chip Shortage

China-Taiwan standoff might increase Global chip shortage

After Nancy Pelosi’s visit to Taiwan, Chinese aircraft are violating Taiwan’s airspace. The escalation made TSMC’s chairman go public and threaten the world with consequences. Can this move by China fuel a global chip shortage?

Another bill bites the dust

The Bill had faced heavy criticism from different stakeholders -citizens, tech firms, political parties since its inception

So long, Spotify

‘TikTok Music’ is set to take over the online streaming space, but there exists an app that has silently established itself in the Indian market.