Google Introduces New Machine Learning Tool To Boost Digital Pathology

Machine learning (ML) is now capable of assisting medical professionals in diagnosing critical cases like diabetic eye disease and metastatic breast cancer.

No matter how improved the algorithms are, they will be useless unless a doctor finds the data sensible. So how doctors interact with information can be crucial determinants in the utility of ML technology.


Sign up for your weekly dose of what's up in emerging technology.

Traditionally pathologists take reference of their colleagues and other sources like textbooks to examine the problem at hand. It can be an image of a cancer scan or of that of blunt trauma. Modern-day technology enables the computers to skim through thousands of images and retrieve information which is the closest to the query. For example, the reverse image search option in Google images. 

Now, there is widespread adoption of ‘digital pathology’. The clinicians can now examine the data received as images on a computer. And, can draw insights from it making the whole process of diagnosis relatively easy.

Download our Mobile App

On these lines, a group of researchers at Google artificial intelligence department have teamed up to introduce machine learning tools and methods to propel the adoption of digital pathology:

  • Similar image search for histopathology – SMILY
  • Human-centered tools for coping with imperfect algorithms during medical decision-making 

Similar Medical Images Like Yours (SMILY)

The first step in developing SMILY was to apply a deep learning model, trained using 5 billion natural, non-pathology images (for example, dogs, trees, man-made objects, etc.) During the training, the network learned to distinguish between similar and dissimilar images by comparing the embeddings. 

Embedding is a summarization of the numerical vector which is a compression of all the images fed into the deep learning models during training.

SMILY allows a user to select a region of interest, and obtain visually-similar matches. As can be seen in the example below where selecting a small region in a slide will enable SMILY to efficiently search a database of billions of cropped images in a few seconds. 

Because pathology images can be viewed at different magnifications (zoom levels), SMILY automatically searches images at the same magnification as the input image. Though these machine learning tools for similar image retrieval drastically reduce the time taken through traditional methods, they fall short on understanding the intent of the user or searcher.

A doctor can look at the same report for different things and it varies from case to case. To make SMILY more interactive, the researchers refined the tool by enabling end-users to express what similarity means on-the-fly: 

  1. Refine-by-region allows pathologists to crop a region of interest within the image, limiting the search to just that region 
  2. Refine-by-example gives users the ability to pick a subset of the search results and retrieve more results like those
  3. Refine-by-concept sliders can be used to specify that more or less of a clinical concept be present in the search results (e.g., fused glands)

The above concepts need not be built into a machine learning model. Instead, this method allows end-users to create new concepts post-hoc, customising the search algorithm towards concepts they find important for each specific use case.

via Google AI blog

The above illustration depicts how interactive the tool is when augmented with refinement.

SMILY: A Clinical Approach

The study conducted by the researchers was published in Nature journal and the results show that the pathologists have found this new tool useful in decision making like keeping a track of the likelihood of a hypothesis.

Tools such as these make the doctors more inclusive in the digital process and eliminate the passivity that the existing methods bring with them. SMILY strikes the right balance by equipping the doctors with domain expertise while allowing them to customise the tool for their own benefit.

Know more about the tool.

More Great AIM Stories

Ram Sagar
I have a master's degree in Robotics and I write about machine learning advancements.

AIM Upcoming Events

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Early Bird Passes expire on 10th Feb

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

Top BI tools for Mainframes

Without BI, organisations will not be able to dominate with data-driven decision-making but focus on experiences, intuition, and gut feelings.