Article complete

Google Introduces KTN to Solve Label Scarcity in Heterogeneous Graphs

Knowledge Transfer Network (KTN) outperforms other transfer learning methods by up to 140%.

Google Introduces KTN to Solve Label Scarcity in Heterogeneous Graphs

Key takeaways

  • Google has introduced the Knowledge Transfer Network (KTN) to address label scarcity in heterogeneous graphs.
  • KTN excels in transferring knowledge from abundant label types to those with no labels, utilising relational information effectively.
  • The model significantly outperforms existing transfer learning methods, achieving improvements of up to 140% in zero-shot learning tasks.
  • Heterogeneous graphs face challenges due to label imbalance, limiting the applicability of traditional HGNNs.
  • KTN consistently surpasses eight state-of-the-art methods in various zero-shot transfer learning tasks.

Takeaways drafted with AI from the article text.

Last week, Google researchers introduced Knowledge Transfer Network (KTN), the first cross-type transfer learning method designed for Heterogeneous graph neural networks (HGNN). With KTN the richness of heterogeneous graphs via HGNNs regardless of label scarcity can be fully exploited. The model was proposed in the paper, “Zero-shot Transfer Learning within a Heterogeneous Graph via Knowledge Transfer Networks”, at NeurIPS 2022. 

KTN transfers knowledge from label-abundant node types to zero-labeled node types using the rich relational information given in a Heterogenous Graph (HG). The researchers pre-trained a HGNN model without needing to fine-tune it. KTN outperforms other transfer learning methods by up to 140% on zero-shot learning tasks and improves existing models by 24% or more.

Various ecosystems can be presented as heterogeneous graphs. HGNNs summarize heterogeneous graph information into representations. However, label scarcity of certain nodes act as a barrier to use HGNNs. The neural nets learn node embeddings by summarizing each node’s relationships into a vector. However, in real world HGs, there is often a label imbalance issue due to different node types. This means that label-scarce node types cannot exploit HGNNs, which hampers applicability of HGNNs.

To examine the effectiveness of KTNs, the researchers ran 18 different zero-shot transfer learning tasks on two public heterogeneous graphs. They compared KTN with eight state-of-the-art transfer learning methods and KTN consistently outperforms all baselines on all tasks, beating transfer learning baselines by up to 140%. Most importantly, KTN can be applied to most HGNN models that have node and edge type-specific parameters and improve their zero-shot performance on target domains. KTN improves accuracy on zero-labeled node types across six different HGNN models by up to 190%.

Published

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

Google · 5 stories
  1. Google's New Technique MorphNet Can Build Smaller, Faster Neural Networks
  2. Google DeepMind announced an early warning system for novel AI risks
  3. You are here: Google Introduces KTN to Solve Label Scarcity in Heterogeneous Graphs
  4. Google Unveils Whitepaper on Generative AI Agents
  5. Google and OpenAI Battle for Math Olympiad Glory
About the author

Contributor

All stories
Belamy · Daily

The day’s AI news, before your first meeting.