Article complete

OpenAI Uses Technique Created By Indian Developers

MRL is a cool embedding representation technique that encodes information at coarse-to-fine granularities in a single vector.

The India Gate monument is featured with a vibrant orange and yellow background, highlighting a central logo.

Key takeaways

  • OpenAI integrates the Matryoshka Representation Learning technique developed by Indian researchers into its embedding models.
  • The new models include a smaller, efficient text-embedding-3-small and a powerful text-embedding-3-large.
  • Researchers Aditya Kusupati and Pratik Jain emphasize the importance of naming their technique in OpenAI's release.
  • Matryoshka Representation Learning enables flexible usage and native support for shortening embeddings.
  • OpenAI's recent updates indicate a growing collaboration with international research talent, particularly from India.

Takeaways drafted with AI from the article text.

Interestingly, the technique behind the embedding has been developed by Indian developers Aditya Kusupati, a researcher at Google and Pratik Jain, a Senior Staff Research Scientist at Google who recently published a paper titled 'Matryoshka Representation Learning.'

"Wish OpenAI had referred to Matryoshka embeddings (or nested embeddings, as we call them in the paper and presentations) instead of avoiding any of the names we have mentioned in the paper," expressed Prateek Jain, one of the authors of the paper on X.

Later on, Owen Campbell-Moore, APIs PM at OpenAI, acknowledged that OpenAI did train on MRL. 'Hey Prateek! We did train this based on MRL - I was responsible for the blog post, and it’s my mistake for not thinking or remembering to cite. We’re updating the blog post to add a citation now!' Moore wrote on X." 

OpenAI edited the blog post mentioning the contribution of Prateek Jain and Aditya Kusupati’s paper. 

What is MRL?

MRL is a cool embedding representation technique that encodes information at coarse-to-fine granularities in a single vector. MRL trains a single high-dimensional vector to encapsulate information at different granularities, akin to nesting dolls. It draws inspiration from the Russian nesting dolls, Matryoshka, where smaller dolls are encased within larger ones.

MRL adapts to various downstream tasks without modifying the original representation, saving computational resources and avoiding the need for separate models for each task.

Published Updated

Spotted an error? Tell our editors. Read our editorial standards and corrections.

About the author

Tech Journalist

Siddharth is a media graduate who loves to explore tech through journalism and putting forward ideas worth pondering about in the era of artificial intelligence.

All stories
Belamy · Daily

The day’s AI news, before your first meeting.