Google Introduces KTN to Solve Label Scarcity in Heterogeneous Graphs

Knowledge Transfer Network (KTN) outperforms other transfer learning methods by up to 140%.
Listen to this story

Last week, Google researchers introduced Knowledge Transfer Network (KTN), the first cross-type transfer learning method designed for Heterogeneous graph neural networks (HGNN). With KTN the richness of heterogeneous graphs via HGNNs regardless of label scarcity can be fully exploited. The model was proposed in the paper, “Zero-shot Transfer Learning within a Heterogeneous Graph via Knowledge Transfer Networks”, at NeurIPS 2022

KTN transfers knowledge from label-abundant node types to zero-labeled node types using the rich relational information given in a Heterogenous Graph (HG). The researchers pre-trained a HGNN model without needing to fine-tune it. KTN outperforms other transfer learning methods by up to 140% on zero-shot learning tasks and improves existing models by 24% or more.

Various ecosystems can be presented as heterogeneous graphs. HGNNs summarize heterogeneous graph information into representations. However, label scarcity of certain nodes act as a barrier to use HGNNs. The neural nets learn node embeddings by summarizing each node’s relationships into a vector. However, in real world HGs, there is often a label imbalance issue due to different node types. This means that label-scarce node types cannot exploit HGNNs, which hampers applicability of HGNNs.

To examine the effectiveness of KTNs, the researchers ran 18 different zero-shot transfer learning tasks on two public heterogeneous graphs. They compared KTN with eight state-of-the-art transfer learning methods and KTN consistently outperforms all baselines on all tasks, beating transfer learning baselines by up to 140%. Most importantly, KTN can be applied to most HGNN models that have node and edge type-specific parameters and improve their zero-shot performance on target domains. KTN improves accuracy on zero-labeled node types across six different HGNN models by up to 190%.

Download our Mobile App

Tasmia Ansari
Tasmia is a tech journalist at AIM, looking to bring a fresh perspective to emerging technologies and trends in data science, analytics, and artificial intelligence.

Subscribe to our newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day.
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Our Upcoming Events

15th June | Bangalore

Future Ready | Lead the AI Era Summit

15th June | Online

Building LLM powered applications using LangChain

17th June | Online

Mastering LangChain: A Hands-on Workshop for Building Generative AI Applications

20th June | Bangalore

Women in Data Science (WiDS) by Intuit India

Jun 23, 2023 | Bangalore

MachineCon 2023 India

26th June | Online

Accelerating inference for every workload with TensorRT

MachineCon 2023 USA

Jul 21, 2023 | New York

Cypher 2023

Oct 11-13, 2023 | Bangalore

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

Can Apple Save Meta?

The iPhone kicked off the smartphone revolution and saved countless companies. Could the Pro Reality headset do the same for Meta?