OpenAI Embedding Models for RAG

OpenAI’s embedding models are reliable text embedders for RAG, especially when you want an easy API and configurable dimensions. Their real limits are text-only scope, token caps, and higher cost than smaller or open models. As of July 2026.

What are the OpenAI embedding models?

As of July 2026, the current text models are text-embedding-3-small and text-embedding-3-large, with text-embedding-ada-002 still present in OpenAI’s docs as the legacy reference. OpenAI’s current embeddings guide states the defaults plainly: text-embedding-3-small returns 1536 dimensions and text-embedding-3-large returns 3072. That matters in RAG because the embedding model decides what the index can find, which means model choice changes retrieval quality before reranking or prompting ever begin.

Which OpenAI embedding model should you use for RAG?

Use text-embedding-3-small when cost and throughput matter more than the last few benchmark points; use text-embedding-3-large when multilingual retrieval quality or harder semantic distinctions justify a larger vector and a higher token bill. OpenAI’s January 25, 2024 launch note quantified the gap against ada-002: MIRACL rose from 31.4 to 44.0 for small and 54.9 for large, while MTEB rose from 61.0 to 62.3 for small and 64.6 for large. That makes large the stronger retriever on public benchmarks, but not automatically the right purchase.

The broader cross-provider decision still belongs to how to choose an embedding model for RAG. This page’s narrower answer is that OpenAI gives you a cheap default and a better large model, not a single universal winner.

Are OpenAI embeddings free, and how much do they cost?

No. OpenAI embeddings are paid per input token. In OpenAI’s January 25, 2024 launch note, text-embedding-3-small is listed at $0.00002 per 1K input tokens and text-embedding-3-large at $0.00013 per 1K input tokens. OpenAI’s current embeddings guide translates that into a rough planning shortcut: about 62,500 pages per dollar for small and 9,615 pages per dollar for large, assuming roughly 800 tokens per page. Those are OpenAI’s published figures, not this site’s estimates, and you should verify the live pricing page before you budget in July 2026.

The embedding bill is only one part of a RAG system’s cost. Total spend still depends on chunk count, re-embedding frequency, storage, retrieval traffic, and generation. That full accounting lives on what a RAG system actually costs to run.

How do OpenAI embedding dimensions affect storage and retrieval?

OpenAI’s two text embedders default to 1536 and 3072 dimensions, and both support shorter output through the dimensions parameter. In RAG, that changes the index bill directly: wider vectors cost more to store and compare, while shorter vectors trade some retrieval quality for lower vector-store cost. OpenAI’s January 2024 launch note makes a strong version of that claim: text-embedding-3-large shortened to 256 dimensions still outperformed unshortened ada-002 at 1536 on MTEB.

The practical footgun is not the API parameter, but manual slicing. OpenAI’s current embeddings guide says that if you change dimensions after generation, you should re-normalize before search. That detail matters because OpenAI’s embeddings are normalized to length 1; once you cut them manually, you must restore the vector shape your similarity math expects. The full truncation and storage trade-off lives on embedding dimensions, truncation and Matryoshka.

How do you use OpenAI embeddings in a RAG pipeline?

Use OpenAI embeddings as the search-stage encoder, not as the vector store itself. The sequence is simple: chunk documents, call client.embeddings.create, store the vectors in a vector database, embed the query with the same model, then rank by cosine similarity or dot product. OpenAI’s cookbook material, summarized in the extracted DeepWiki page, uses this as a standard Search-Ask RAG pattern: search your text library by embedding similarity, then insert the retrieved text into the answer prompt.

OpenAI’s current embeddings guide also notes that cosine similarity and Euclidean distance produce identical rankings for these normalized vectors, while cosine can be computed as a dot product. If you use LangChain, the current OpenAIEmbeddings integration shows the same flow in a higher-level wrapper: embed documents on indexing, embed queries on retrieval, and pass the vectors into a retriever. What embeddings are doing in the pipeline belongs to embeddings in RAG; where you store them belongs to the vector database profiles.

What limits matter when you use OpenAI embeddings in production?

The practical limits are input size, batch size, and model scope. Microsoft’s Azure OpenAI how-to, last updated on 2026-07-22, states the current hard ceilings explicitly: a maximum input length of 8192 tokens for the current embedding models, a maximum array size of 2048 inputs in one request, and a maximum aggregate of 300000 input tokens across a single embeddings request. Requests above those limits fail with HTTP 400. Those numbers are operationally useful even if you call OpenAI directly, because they tell you where batching and document splitting stop being optional.

The softer limit is scope. The text-embedding-3 models are text embedders, not general multimodal retrievers, so if your RAG pipeline must index images, video, or mixed media, competing models have a wider native surface. The other non-obvious limit is migration cost: if you change model family or stored vector width, you cannot safely mix old and new vectors in one index, so a real switch means re-embedding the corpus. If your documents routinely exceed the model’s input cap, the cookbook pattern is chunking or chunk-and-average rather than “just send longer text.”

Are OpenAI embeddings better than Cohere, Voyage, or open-source models?

No single provider wins every retrieval shape. OpenAI remains a strong default for text RAG because the API is simple, the published multilingual retrieval scores are solid, and the current models expose configurable dimensions without extra serving work. But other models can beat it on things OpenAI’s profile does not target. A March 2026 independent benchmark by Cheney Zhang measured OpenAI 3-large at 0.967 on the post’s cross-lingual round and a perfect score within its tested needle-in-a-haystack window, while Voyage Multimodal 3.5 led that post’s compression-oriented MRL round at 0.874 versus OpenAI’s 0.762, and Gemini Embedding 2 led its cross-lingual and longer-window tasks.

The honest summary is narrower than a winner-take-all list: OpenAI is strongest when you want a text-first hosted default with low integration friction. It is weaker when you need multimodal retrieval, very long input windows, or a model chosen specifically for compression resilience. That is why this page stops at the OpenAI-specific trade-offs and routes the broader provider choice to embedding model selection.