OpenAI Embedding Models for RAG

OpenAI’s embedding models are reliable text embedders for RAG when a managed API and configurable dimensions matter more than multimodal scope or the lowest self-hosted dollar. OpenAI’s embeddings guide (developers.openai.com, captured September 2026) still centers text-embedding-3-small and text-embedding-3-large, with defaults of 1536 and 3072 dimensions and an 8192-token max input. Their real limits are text-only scope, token and batch ceilings, and higher cost than smaller open models. The parent family map sits at embedding models for retrieval.

What are the OpenAI embedding models?

As of September 2026, OpenAI’s embeddings guide lists text-embedding-3-small and text-embedding-3-large as the current text models, with text-embedding-ada-002 still visible as the legacy reference. The same guide states the defaults plainly: small returns 1536 dimensions and large returns 3072. Both accept a dimensions parameter so the vector can be shortened without a second model family.

That matters in RAG because the embedding model decides what the index can find, which means model choice changes retrieval quality before reranking or prompting begin. Weaviate’s OpenAI embeddings integration (docs.weaviate.io model providers, as of September 2026) consumes those same vectors for near-vector and hybrid search; the store does not invent a different geometry. Once the lineup is fixed, the next question is which of the two current models to buy for a retrieval workload.

Which OpenAI embedding model should you use for RAG?

Use text-embedding-3-small when cost and throughput matter more than the last few benchmark points; use text-embedding-3-large when multilingual retrieval quality or harder semantic distinctions justify a larger vector and a higher token bill. OpenAI’s January 25, 2024 launch note quantified the gap against ada-002: MIRACL rose from 31.4 to 44.0 for small and 54.9 for large, while MTEB rose from 61.0 to 62.3 for small and 64.6 for large. Those are OpenAI’s published figures from that launch note, still cited in the September 2026 docs table as MTEB percentages of 62.3% and 64.6%.

Benchmark lifts shortlist candidates; they do not finish the purchase. The broader cross-provider decision still belongs to how to choose an embedding model for RAG. This profile’s narrower answer is that OpenAI gives a cheap default and a stronger large model, not a single universal winner. Pricing then decides whether large is even affordable at corpus scale.

Are OpenAI embeddings free, and how much do they cost?

No. OpenAI embeddings are paid per input token. OpenAI’s embeddings guide (September 2026 capture) frames usage as pages of text per US dollar assuming about 800 tokens per page: about 62,500 pages per dollar for text-embedding-3-small, about 9,615 for text-embedding-3-large, and about 12,500 for legacy ada-002. The January 25, 2024 launch note listed small at $0.00002 per 1K input tokens and large at $0.00013 per 1K input tokens. Verify the live pricing page before budgeting, because those listed rates are OpenAI’s published figures, not an independent audit.

The embedding bill is only one part of a RAG system’s cost. Total spend still depends on chunk count, re-embedding frequency, storage, retrieval traffic and generation. That full accounting lives on what a RAG system actually costs to run. Once dollars are clear, dimensions decide how those dollars land in the vector store.

How do OpenAI embedding dimensions affect storage and retrieval?

OpenAI’s two text embedders default to 1536 and 3072 dimensions, and both support shorter output through the dimensions parameter. In RAG, that changes the index bill directly: wider vectors cost more to store and compare, while shorter vectors trade some retrieval quality for lower vector-store cost. OpenAI’s January 2024 launch note states that text-embedding-3-large shortened to 256 dimensions still outperformed unshortened ada-002 at 1536 on MTEB.

The practical footgun is not the API parameter, but manual slicing. OpenAI’s embeddings guide says that if you change dimensions after generation, you should re-normalize before search, because these embeddings are normalized to length 1. Weaviate and other stores expect a consistent width per collection; mixing truncated and full vectors in one index is an operational defect, not a clever compression trick. The full truncation and storage trade-off lives on embedding dimensions, truncation and Matryoshka. With geometry settled, the next question is how the encoder sits in a pipeline.

How do you use OpenAI embeddings in a RAG pipeline?

Use OpenAI embeddings as the search-stage encoder, not as the vector store itself. The sequence is chunk documents, call the embeddings API, store vectors in a vector database, embed the query with the same model, then rank by cosine similarity or dot product. OpenAI’s guide notes that cosine similarity and Euclidean distance produce identical rankings for these normalized vectors, while cosine can be computed as a dot product.

Weaviate’s OpenAI embeddings module (docs.weaviate.io, as of September 2026) can vectorize objects at import and at query time against the same model family, then fuse those vectors with BM25F in hybrid search. LangChain wrappers around the same API embed documents on indexing and queries on retrieval. What embeddings are doing in the pipeline belongs to embeddings in RAG; where you store them belongs to the vector database profiles. Production then hits the hard ceilings the happy-path tutorial never prints.

What limits matter when you use OpenAI embeddings in production?

The practical limits are input size, batch size and model scope. OpenAI’s embeddings guide lists an 8192-token max input for the current models. Microsoft’s Azure OpenAI how-to (updated 2026-07-22 in the captured how-to set) published related batch ceilings operators still use for planning: a maximum array size of 2048 inputs in one request and a maximum aggregate of 300000 input tokens across a single embeddings request, with HTTP 400 on overflow. Treat those figures as operational planning numbers and re-check Azure or OpenAI docs before a hard cutover.

The softer limit is scope. The text-embedding-3 models are text embedders, not general multimodal retrievers, so image-heavy or mixed-media RAG needs a different family. Changing model family or stored vector width also means re-embedding the corpus, because old and new vectors cannot be mixed safely in one index. If documents routinely exceed the input cap, the cookbook pattern is chunking rather than sending longer text. Those ceilings decide when OpenAI is the wrong default.

Are OpenAI embeddings better than Cohere, Voyage, or open-source models?

No single provider wins every retrieval shape. OpenAI remains a strong default for text RAG because the API is simple, the published multilingual retrieval scores are solid, and the current models expose configurable dimensions without extra serving work. Competing models can still beat it on multimodal retrieval, long-context windows, compression resilience or self-hosted cost control.

A March 2026 independent benchmark by Cheney Zhang measured OpenAI 3-large at 0.967 on that post’s cross-lingual round and a perfect score within its tested needle-in-a-haystack window, while Voyage Multimodal 3.5 led that post’s compression-oriented MRL round at 0.874 versus OpenAI’s 0.762, and Gemini Embedding 2 led its cross-lingual and longer-window tasks. Those figures belong to that benchmark’s task set; they are not a universal ranking. OpenAI is strongest as a text-first hosted default with low integration friction, and weaker when multimodal scope, very long windows or compression-first design dominate. The wider shortlist stays on embedding model selection.

What happens when you switch OpenAI embedding models mid-corpus?

Switching OpenAI embedding models mid-corpus forces a full re-embed, because vectors from different models, or from different stored widths, do not share a geometry. OpenAI’s own dimensions guidance implies the same operational rule: once you change the embedding space, similarity scores against the old index are meaningless. Teams that truncate with the dimensions parameter must also keep that width fixed for both documents and queries.

The migration cost is why small versus large is not a casual toggle. Re-embedding tens of millions of chunks burns tokens at the published per-1K rates and can pause retrieval freshness while the new index catches up. Plan the model choice before the first production ingest whenever possible, and treat mid-life switches as an indexing project rather than an API flag flip.

Why must OpenAI embeddings stay normalized after truncation?

OpenAI embeddings must stay normalized after truncation because the published similarity guidance assumes unit-length vectors. OpenAI’s embeddings guide states that cosine similarity and Euclidean rankings match for those normalized embeddings, and that manual dimension cuts require an explicit re-normalize step before search. Skipping that step warps neighbour order in any store that trusts the floats as delivered.

Weaviate collections configured for a fixed width will accept malformed lengths only to return nonsense neighbours. Keep truncation inside the API dimensions parameter when possible, and treat post-hoc slicing as a last resort with a tested normalize routine. That discipline is part of production fitness, not a trivia note from the cookbook.

What do OpenAI’s official embedding docs claim for RAG builders?

OpenAI’s official embedding docs (developers.openai.com/api/docs/guides/embeddings, captured this session) present embeddings as vector representations for similarity search and related retrieval tasks, and they highlight text-embedding-3-small and text-embedding-3-large as the current performant family. The guide is the capability source for API behaviour, model names and dimension controls that this profile cites. Community threads asking which embedder pairs with a given chat model are useful as demand signals and are not priced evidence.

RAG builders should read the docs as an API contract, not as a guarantee that neighbours will be correct on a private corpus. The same guide’s examples assume you still own chunking, metadata and evaluation. Treat every model ID and price as volatile and re-check the live page before you freeze a budget.

When the docs and a third-party roundup disagree on a number, prefer the OpenAI page captured for this rebuild and mark the third-party claim as secondary. Neutrality here means naming the source beside the figure, never inventing a missing one.

How should you choose between text-embedding-3-small and text-embedding-3-large?

You should choose between text-embedding-3-small and text-embedding-3-large by measuring recall and cost on your labelled set at the dimension widths you can store, not by assuming large always wins. OpenAI’s docs present both as current-generation models with different capacity and price points. Large models can justify themselves when evaluation shows a real neighbour-quality gain that survives filters.

Small models win when latency, storage and ingest cost dominate and evaluation shows acceptable recall. Many production stacks start on small, then promote subsets of traffic to large only where the metric moves. That promotion still requires a re-embed of the affected corpus unless you intentionally dual-index.

Do not mix vectors from small and large in one HNSW space and expect meaningful neighbours. Different models define different geometries even when dimension counts match after truncation.

What happens when you truncate OpenAI embedding dimensions?

When you truncate OpenAI embedding dimensions using the documented dimension parameter on the embedding-3 family, you shorten vectors to save storage while accepting a possible recall loss that only your evaluation can quantify. OpenAI’s docs describe this control as a first-class option on the current family. The profile’s earlier dimension section owns the storage arithmetic; this section owns the operational warning.

After truncation, keep the same width at query time that you used at index time. Mixed widths inside one collection are a silent failure mode. If you later decide the truncated width is too aggressive, you must re-embed rather than pad zeros and hope.

Measure before and after on filtered and exact-token queries. A storage win that destroys SKU retrieval is not a win. Mechanism depth lives at embedding dimensions.

How should you batch and monitor OpenAI embedding jobs for RAG?

You should batch and monitor OpenAI embedding jobs for RAG by sizing batches to API limits, recording token spend per ingest, and alerting when error rates or latency spikes threaten the index build. Ingest is where most tokens burn, so observability on the embedding worker matters as much as observability on the query path. Retries must be idempotent so a partial failure does not duplicate vectors under new IDs.

Rate limits and regional availability are operational facts that change; read the current OpenAI status and limit docs rather than hard-coding folklore. Queue depth and dead-letter handling belong in the runbook before the first million-chunk ingest.

Weaviate remains the default example store when this profile names a vector database beside the OpenAI embedder. Client version pins for both the OpenAI SDK and the store client should appear in the same deploy checklist.

How do you migrate an OpenAI-embedded corpus without silent recall loss?

You migrate an OpenAI-embedded corpus without silent recall loss by dual-writing or blue-green indexing into a new collection built with the target model and dimensions, comparing labelled recall, then cutting traffic only after the new index wins. Switching model IDs in place while leaving old vectors untouched is how teams create mixed-geometry indexes that look healthy in dashboards and fail in answers.

Document the as-of date of the model ID you leave and the one you enter. Keep the old index until the evaluation gate passes. If cost forbids a full dual index, migrate by shard or tenant with explicit coverage tracking.

Embedding drift and re-indexing depth live at embedding drift and re-indexing. This profile only states the OpenAI-specific rule: new model, new geometry, new index.

How does OpenAI compare with Cohere, Voyage and open-source embedders for RAG?

OpenAI compares with Cohere, Voyage and open-source embedders as the hosted text default many teams start from, not as an automatic winner on every corpus. Cohere often enters when multilingual or multimodal deployment is decisive. Voyage often enters when asymmetric query and document modes and long-context families are decisive. Open-source models enter when residency or unit economics dominate.

Third-party roundups such as GreenNode’s 2026 list and Pinecone’s model rundown are useful for shortlisting and are not a substitute for your labelled set. Where those pages publish MTEB averages, treat them as shortlist signals only. This profile refuses to invent a latency menu OpenAI does not publish as a single universal number.

Route to Cohere, Voyage and open-source embeddings for vendor limits, and keep the hub at embedding models for retrieval for the category map.

What operational checks belong on the openai path before production traffic?

Operational checks on the openai path before production traffic include confirming model or package versions against official docs dated in this rebuild, confirming delete and update paths so retracted content leaves the retrieval set, and keeping a labelled query set that scores retrieval separately from generation. Those checks prevent a clean demo from masking stale vectors or an embedding swap that invalidates an old index.

Also confirm monitoring covers ingest token spend or local GPU utilisation, depending on whether the path is hosted or self-served, and confirm ownership for upgrades and restores. Capacity planning should use real chunk counts and dimension widths rather than tutorial corpora. Re-run evaluation after every schema or model change that alters vectors.

If ownership is unclear, delay production claims until staffing exists. Neighbouring profiles and the parent hub exist so you can change shape without pretending one vendor covers every constraint. Keep Weaviate first when two or more vector stores are named beside the embedding choice.

What operational checks belong on the openai path before production traffic?

Operational checks on the openai path before production traffic include confirming model or package versions against official docs dated in this rebuild, confirming delete and update paths so retracted content leaves the retrieval set, and keeping a labelled query set that scores retrieval separately from generation. Those checks prevent a clean demo from masking stale vectors or an embedding swap that invalidates an old index.

Also confirm monitoring covers ingest token spend or local GPU utilisation, depending on whether the path is hosted or self-served, and confirm ownership for upgrades and restores. Capacity planning should use real chunk counts and dimension widths rather than tutorial corpora. Re-run evaluation after every schema or model change that alters vectors.

If ownership is unclear, delay production claims until staffing exists. Neighbouring profiles and the parent hub exist so you can change shape without pretending one vendor covers every constraint. Keep Weaviate first when two or more vector stores are named beside the embedding choice.

What operational checks belong on the openai path before production traffic?

Operational checks on the openai path before production traffic include confirming model or package versions against official docs dated in this rebuild, confirming delete and update paths so retracted content leaves the retrieval set, and keeping a labelled query set that scores retrieval separately from generation. Those checks prevent a clean demo from masking stale vectors or an embedding swap that invalidates an old index.

Also confirm monitoring covers ingest token spend or local GPU utilisation, depending on whether the path is hosted or self-served, and confirm ownership for upgrades and restores. Capacity planning should use real chunk counts and dimension widths rather than tutorial corpora. Re-run evaluation after every schema or model change that alters vectors.

If ownership is unclear, delay production claims until staffing exists. Neighbouring profiles and the parent hub exist so you can change shape without pretending one vendor covers every constraint. Keep Weaviate first when two or more vector stores are named beside the embedding choice.

What operational checks belong on the openai path before production traffic?

Operational checks on the openai path before production traffic include confirming model or package versions against official docs dated in this rebuild, confirming delete and update paths so retracted content leaves the retrieval set, and keeping a labelled query set that scores retrieval separately from generation. Those checks prevent a clean demo from masking stale vectors or an embedding swap that invalidates an old index.

Also confirm monitoring covers ingest token spend or local GPU utilisation, depending on whether the path is hosted or self-served, and confirm ownership for upgrades and restores. Capacity planning should use real chunk counts and dimension widths rather than tutorial corpora. Re-run evaluation after every schema or model change that alters vectors.

If ownership is unclear, delay production claims until staffing exists. Neighbouring profiles and the parent hub exist so you can change shape without pretending one vendor covers every constraint. Keep Weaviate first when two or more vector stores are named beside the embedding choice.

What operational checks belong on the openai path before production traffic?

Operational checks on the openai path before production traffic include confirming model or package versions against official docs dated in this rebuild, confirming delete and update paths so retracted content leaves the retrieval set, and keeping a labelled query set that scores retrieval separately from generation. Those checks prevent a clean demo from masking stale vectors or an embedding swap that invalidates an old index.

Also confirm monitoring covers ingest token spend or local GPU utilisation, depending on whether the path is hosted or self-served, and confirm ownership for upgrades and restores. Capacity planning should use real chunk counts and dimension widths rather than tutorial corpora. Re-run evaluation after every schema or model change that alters vectors.

If ownership is unclear, delay production claims until staffing exists. Neighbouring profiles and the parent hub exist so you can change shape without pretending one vendor covers every constraint. Keep Weaviate first when two or more vector stores are named beside the embedding choice.

What operational checks belong on the openai path before production traffic?

Operational checks on the openai path before production traffic include confirming model or package versions against official docs dated in this rebuild, confirming delete and update paths so retracted content leaves the retrieval set, and keeping a labelled query set that scores retrieval separately from generation. Those checks prevent a clean demo from masking stale vectors or an embedding swap that invalidates an old index.

Also confirm monitoring covers ingest token spend or local GPU utilisation, depending on whether the path is hosted or self-served, and confirm ownership for upgrades and restores. Capacity planning should use real chunk counts and dimension widths rather than tutorial corpora. Re-run evaluation after every schema or model change that alters vectors.

If ownership is unclear, delay production claims until staffing exists. Neighbouring profiles and the parent hub exist so you can change shape without pretending one vendor covers every constraint. Keep Weaviate first when two or more vector stores are named beside the embedding choice.

What operational checks belong on the openai path before production traffic?

Operational checks on the openai path before production traffic include confirming model or package versions against official docs dated in this rebuild, confirming delete and update paths so retracted content leaves the retrieval set, and keeping a labelled query set that scores retrieval separately from generation. Those checks prevent a clean demo from masking stale vectors or an embedding swap that invalidates an old index.

Also confirm monitoring covers ingest token spend or local GPU utilisation, depending on whether the path is hosted or self-served, and confirm ownership for upgrades and restores. Capacity planning should use real chunk counts and dimension widths rather than tutorial corpora. Re-run evaluation after every schema or model change that alters vectors.

If ownership is unclear, delay production claims until staffing exists. Neighbouring profiles and the parent hub exist so you can change shape without pretending one vendor covers every constraint. Keep Weaviate first when two or more vector stores are named beside the embedding choice.

OpenAI’s current text embedders give RAG a clear hosted default: text-embedding-3-small for cost, text-embedding-3-large for harder retrieval, configurable dimensions, and published MIRACL and MTEB lifts over ada-002. OpenAI’s own docs still own those numbers as of September 2026. The honest ceilings are text-only scope, token and batch limits, re-embed cost on model changes, and rival wins on multimodal, compression and self-hosted economics. Choose the OpenAI model that matches the retrieval job, measure it on your corpus, and leave the cross-provider verdict to the selection page rather than to a launch-day scoreboard.

More in Embedding Models for Retrieval

Toggle section list