OpenAI recently released new embedding models and API updates, introducing two additions to its lineup: a smaller and highly efficient text-embedding-3-small model, and a larger and more powerful text-embedding-3-large model. 

Interestingly, the technique behind the embedding has been developed by Indian developers Aditya Kusupati, a researcher at Google and Pratik Jain, a Senior Staff Research Scientist at Google who recently published a paper titled ‘Matryoshka Representation Learning.’

“Wish OpenAI had referred to Matryoshka embeddings (or nested embeddings, as we call them in the paper and presentations) instead of avoiding any of the names we have mentioned in the paper,” expressed Prateek Jain, one of the authors of the paper on X.

Later on, Owen Campbell-Moore, APIs PM at OpenAI, acknowledged that OpenAI did train on MRL. ‘Hey Prateek! We did train this based on MRL – I was responsible for the blog post, and it’s my mistake for not thinking or remembering to cite. We’re updating the blog post to add a citation now!’ Moore wrote on X.” 

OpenAI edited the blog post mentioning the contribution of Prateek Jain and Aditya Kusupati’s paper. 

What is MRL?

MRL is a cool embedding representation technique that encodes information at coarse-to-fine granularities in a single vector. MRL trains a single high-dimensional vector to encapsulate information at different granularities, akin to nesting dolls. It draws inspiration from the Russian nesting dolls, Matryoshka, where smaller dolls are encased within larger ones.

MRL adapts to various downstream tasks without modifying the original representation, saving computational resources and avoiding the need for separate models for each task.