Article complete

Now, OpenAI API have text and code embeddings

OpenAI logo against a colorful gradient background, highlighting advancements in text and code embeddings.

Key takeaways

  • OpenAI has launched a new embeddings endpoint in its API for improved semantic search and classification.
  • The embeddings demonstrate a significant performance boost, particularly in code search, outperforming leading models.
  • Embeddings represent text and code in a high-dimensional space, capturing semantic similarities through numerical vectors.
  • Three distinct families of embedding models have been introduced, catering to text similarity, text search, and code search functionalities.

Takeaways drafted with AI from the article text.

OpenAI’s embeddings outperform top models in three standard benchmarks, including a 20% relative improvement in code search. Embeddings are really useful for working with natural language and code.

The embeddings that are numerically similar are also semantically similar. For example, the embedding vector of “canine companions say” will be more similar to the embedding vector of “woof” than “meow.” The new endpoint by OpenAI uses neural network models to map text and code to a vector representation—“embedding” them in a high-dimensional space. Each dimension captures some aspect of the input.

The company has released three families of embedding models for different functionalities including text similarity, text search, and code search. The models take either text or code as input and return an embedding vector.

Text similarity models

The text similarity models provide embeddings that capture the semantic similarity of pieces of text. These models are useful for many tasks including clustering, data visualization, and classification.

Text search models

The text search models provide embeddings that allow large-scale search tasks, such as finding a relevant document among a collection of documents given a text query. The model first embeds for the documents and produces query separately, and then cosine similarity is used to compare the similarity between the query and each document. Such embedding-based search generalize better than word overlap techniques used in the classical keyword search, as it captures the semantic meaning of the text and is also less sensitive to exact phrases or words.

Code search models

Code search models provide code and text embeddings for code search tasks. Given a collection of code blocks, the task is to find the relevant code block for a natural language query. 

Find the embeddings documentation here.

Published

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

API · 3 stories
  1. You are here: Now, OpenAI API have text and code embeddings
  2. After Getting Banned in Schools, OpenAI Launches ChatGPT Tool for Teachers
  3. Why ChatGPT Business is Destined to Fail
About the author

Contributor

All stories
Belamy · Daily

The day’s AI news, before your first meeting.