Advertisement

What Are Prototypical Networks?

Prototypical networks are more efficient than the recent meta-learning algorithms, making them an appealing approach to few-shot and zero-shot learning.

Few-shot learning (FSL) or low-shot learning (LSL) is a type of meta-learning. Unlike traditional AI/ML models that require a large amount of data to produce good results, AI/ML models with few-shot learning capability require only a small amount of training data with limited information. 

In few-shot learning, a learner is trained on various related tasks during the meta-training phase to generalise well to unseen tasks with just a few instances during the meta-testing phase. 

Few-shot classification is a method where a classifier must be adapted to accommodate new classes that are not seen in training and given only a few examples of each of these classes. In recent years, AI researchers have utilised this method for various applications in computer vision, natural language processing (NLP), acoustic signal processing and more. However, approaches such as re-training the model on new data can lead to overfitting of the model.

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

To mitigate such issues in few-shot classification, a team of researchers from Twitter and the University of Toronto has introduced prototypical networks. Prototypical networks learn a metric space where classification can be executed by computing distances to prototype representations of each class.

What are prototypical networks?

Prototypical networks are based on the concept that there exists an embedding in which several points cluster around a single prototype representation for each class. It aims to learn per-class prototypes based on sample averaging in the feature space.


Download our Mobile App



To be more specific, prototypical networks compute each class’s M-dimensional representation or prototype through an embedding function with learnable parameters. Also, each prototype is the mean vector of the embedded support points belonging to its class. 

Prototypical networks are more efficient than the recent meta-learning algorithms, making them an appealing approach to few-shot and zero-shot learning. They learn a metric space in which classification can be performed by computing distances to prototype representations of each class. Prototypical networks are related to the neural statistician from the generative modelling literature that extends the variational autoencoder to learn generative models of datasets rather than just the individual points.

Prototypical networks vs matching networks

In recent years, approaches such as matching networks have made significant progress in few-shot learning. Matching networks use an attention mechanism over a learned embedding of the labelled set of examples to predict classes for the unlabelled points.

Both prototypical networks and matching networks were introduced to mitigate the various shortcomings of the few-shot learning technique. The differences between both the networks are as follows-

  • Compared to the recent approaches for few-shot learning, such as matching networks, prototypical networks reflect a more simplistic inductive bias, which is beneficial in the limited-data regime and achieve exceptional results.
  • Prototypical networks vary from matching networks in the few-shot case with equivalence in the one-shot scenario. This means that, in the case of one-shot learning, since there is only one support point per class, matching networks and prototypical networks become equivalent.
  • Matching networks produce a weighted nearest neighbour classifier given the support set, while prototypical networks produce a linear classifier when squared Euclidean distance is used.

Wrapping up

Prototypical Networks is a simple method for few-shot learning based on the idea that one can represent each class by the mean of its examples in a representation space learned by a neural network. The approach is more efficient and far simpler than recent meta-learning approaches and produces state-of-the-art results even without sophisticated extensions developed for matching networks.

More Great AIM Stories

Ambika Choudhury
A Technical Journalist who loves writing about Machine Learning and Artificial Intelligence. A lover of music, writing and learning something out of the box.

AIM Upcoming Events

Regular Passes expire on 3rd Mar

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Early Bird Passes expire on 17th Feb

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, Virtual
Deep Learning DevCon 2023
27 May, 2023

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox
AIM TOP STORIES

What went wrong with Meta?

Many users have opted out of Facebook and other applications tracking their activities now that they must explicitly ask for permission.