A vector database is the piece that turns text, images, audio, and code into mathematical coordinates (embeddings) and makes it possible to find what is semantically similar, not just what has the same word. It is the technical foundation of RAG, semantic search, smart recommendation, deduplication, and almost everything that goes beyond keyword search in modern AI applications.
This guide explains what an embedding is, how a vector DB works, how to choose among Pinecone, Qdrant, Weaviate, Milvus, pgvector, and Chroma, how to design indexes for scale, and how to avoid the mistakes that wreck RAG quality in practice.
What an embedding is and why it matters
An embedding is a vector of numbers (typically 768, 1024, 1536, or 3072 dimensions) that represents the meaning of a piece of content. Texts that say the same thing, even with different words, end up close in the vector space. "How do I cancel my subscription" and "I want to stop paying" will have neighboring embeddings, even if they share no keyword.
Embeddings are generated by specific models: OpenAI text-embedding-3, Voyage, Cohere Embed, BGE, Nomic Embed, and variants specialized by language and domain. The choice of embedding model has a huge impact on RAG quality, more than many people imagine.
What a vector database does
Three operations are the core. 1. Ingestion: receive a document, split it into chunks, generate an embedding for each chunk, store it with metadata. 2. Similarity search: given a query vector, return the k nearest vectors by cosine distance, inner product, or Euclidean distance. 3. Hybrid filtering: combine vector similarity with traditional filters (date, author, category, ACL) for useful results.
Underneath, the engine uses algorithms like HNSW, IVF, ScaNN, or DiskANN for approximate indexing that scales to billions of vectors with millisecond latency. Exact search in high dimensions is expensive; the point of a vector DB is to deliver a near-exact result at a viable cost.
Comparison of the main options in 2026
Pinecone: managed, simple, scales easily, high cost from a certain volume on. Good for getting started fast. Qdrant: open-source, self-host or cloud, excellent performance, powerful filters, support for rich payloads. A community favorite. Weaviate: open-source, multi-modal, native GraphQL, integration with several sources. Milvus: open-source, focused on scale (billions of vectors), a strong Chinese community, more complex to operate. pgvector: a Postgres extension. It gained immense traction in 2025 by letting you reuse Postgres as a vector DB, with everything that brings (transactions, ACID, familiar tooling). Chroma: lightweight, focused on local development and prototypes, simple to spin up.
The choice depends on three axes: volume (thousands vs billions of vectors), operation (managed vs self-host), and requirements (complex filters, multi-tenant, ACL). For most enterprise cases in 2026, pgvector or Qdrant handle it with room to spare.
Chunking strategy
The way you split a document into chunks determines RAG quality more than any other choice. Small chunks (256-512 tokens): high precision, limited context, risk of losing cohesion. Medium chunks (512-1024 tokens): a good balance for most cases. Large chunks (1024-2048 tokens): high cohesion, higher cost, dilutes the signal.
Advanced patterns that improve results: structure-based chunking (paragraph, section, code), not blind counting; overlap between chunks to preserve context; parent-child, indexing small chunks but returning the larger parent chunk; contextual chunks, adding the document title and breadcrumb to each chunk.
Hybrid: vector + keyword
Purely vector search falls short on specific queries with exact terms (product name, code, acronym). Purely keyword search falls short on queries with paraphrasing. The modern path is hybrid: combining BM25 (keyword) and vector similarity, with a final re-ranking. Tools like Qdrant, Weaviate, and Elasticsearch already offer this natively.
In serious RAG, adding a re-ranker (Cohere Rerank, Voyage Reranker, BGE Reranker) over the initial top-K usually raises precision noticeably, at an acceptable cost.
Operating in production
Four precautions that separate a prototype from production. 1. Retrieval metrics: measure recall@k, precision@k, MRR on representative evaluation sets. 2. Controlled reindexing: swapping the embedding model means reindexing everything. Have a pipeline and an embedding version per document. 3. Backup and disaster recovery: a vector DB is the base of your product; treat it as a critical database. 4. Access control: a vector DB can leak sensitive data if ACL filters are poorly applied. Apply the permission filter in the index, not just in the response.
Cost: common pitfalls
Vector DB cost tends to surprise. The frequent culprits: chunks that are too small multiply the number of vectors; 3072-dimension embeddings cost 2x the storage vs 1536; reindexing due to a model swap doubles the cost temporarily; managed cloud charges per request on top of storage. Good practices: monitor cost per feature (not just total), quantization of vectors in large indexes, tiered storage separating "hot" and "cold" data.
RAG is not just a vector DB
A frequent mistake is confusing RAG with "installing a vector DB". The vector DB is one piece. Good RAG requires an ingestion pipeline that extracts content from PDFs, websites, and internal bases faithfully; smart chunking; embedding generation with the right model; hybrid retrieval with filters and re-ranking; a prompt that injects the context the right way; a citation of the source in the response; continuous evaluation with your own dataset. Skipping any piece breaks quality.
When a vector DB is not the answer
For low volume (up to a few thousand documents), an in-memory cache or a simple index already solves it. For queries with an exact term (id, code, name), traditional search with an inverted index wins. For structured tabular data, SQL remains unbeatable. The rule: use a vector DB where semantics matter and the set is large enough to justify the complexity. Otherwise, keep it simple.