Chroma OSS Β· embedded or server
Niche The easiest on-ramp β embeds in your Python process, persists to disk, and can embed text for you. Choose prototyping and local RAG.
A vector database stores embeddings (plus their metadata) and answers one question extremely fast: which stored vectors are nearest this query vector? That "nearest-neighbour" search is the retrieval engine behind semantic search and RAG. The options range from a one-line library to a distributed cluster β this guide compares them.
Finding the closest vectors exactly means comparing the query against every stored vector β O(NΒ·d). Over millions of high-dimensional vectors that's far too slow for interactive search. Vector databases solve this with Approximate Nearest Neighbour (ANN) indexes that trade a tiny bit of accuracy (recall) for enormous speed-ups β plus the un-glamorous-but-essential parts: metadata filtering, persistence, updates, and scaling.
Every vector database, big or small, does the same four jobs:
Keep vectors + metadata (source, tags, timestamps) durably, with add/update/delete.
Build a structure β HNSW graph, IVF clusters, PQ compression β for fast approximate search.
Compare with cosine, dot product or Euclidean β matched to how the embeddings were trained.
Combine vector similarity with metadata filters (source = "kb") and often keyword/hybrid search.
At query time: embed the query with the same model used for indexing, run ANN search for the top-k nearest, apply filters, and return the matches (with their original text/metadata) to feed a model.
HNSW β a navigable small-world graph you greedily walk toward the query; great recall/latency, memory-hungry. IVF β cluster vectors, search only the nearest clusters. PQ (product quantization) β compress vectors to save memory at some accuracy cost. DiskANN β graph indexes that live on SSD for billion-scale sets. Most databases default to HNSW.
The options spread across two axes: how much you run yourself (a library β a managed service) and how much scale you're built for.
No point is "best" β each occupies a sensible niche. The trick is knowing your position: how much ops you'll take on, and how big you'll get.
Niche The easiest on-ramp β embeds in your Python process, persists to disk, and can embed text for you. Choose prototyping and local RAG.
Niche A blazing-fast similarity-search library, not a database β no server, filtering or persistence out of the box. Choose static, read-heavy, research or custom pipelines where you manage storage yourself.
Niche A production-grade server with rich metadata filtering and good performance. Choose balanced self-hosted production with strong filtering.
Niche Feature-rich with first-class hybrid search (vector + keyword) and modules. Choose when hybrid search and built-in modules matter.
Niche Built for massive scale β billions of vectors across a distributed cluster. Choose very large deployments that need horizontal scale.
Niche Fully managed β no infrastructure to run; scales for you. Choose hands-off production when you'd rather not operate a database.
Niche Adds a vector column and ANN search to Postgres β vectors live beside your relational data, filtered with plain SQL. Choose when you're already on Postgres and want one datastore.
Niche Databases you may already run have added vector/kNN search. Choose to avoid a new system when you're already invested in one of them.
The same "store + search" in three kinds of vector store β an embedded library, a managed service, and a SQL extension. Switch tabs; hit Copy.
# Embedded library β it can even embed the text for you.
import chromadb
client = chromadb.PersistentClient("./db")
col = client.get_or_create_collection("docs")
col.add(ids=["1"], documents=["Our refund policyβ¦"], metadatas=[{"src": "kb"}])
print(col.query(query_texts=["how do I get money back"], n_results=3))
# Managed service β you bring the vectors; it runs the index in the cloud.
from pinecone import Pinecone
pc = Pinecone(api_key="...")
idx = pc.Index("docs")
idx.upsert([("1", embedding, {"src": "kb"})])
print(idx.query(vector=query_emb, top_k=3, filter={"src": "kb"}))
-- Postgres extension β vectors are just a column next to your other data.
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE docs (id bigserial, embedding vector(384), src text);
-- nearest neighbours by cosine distance (<=>), with an ordinary WHERE filter
SELECT id FROM docs
WHERE src = 'kb'
ORDER BY embedding <=> $1
LIMIT 3;
| Store | Kind | Hosting | Filtering | Scale | Best for |
|---|---|---|---|---|---|
| Chroma | OSS database | Embedded / server | Yes | Smallβmed | Prototyping, local RAG |
| FAISS | OSS library | In-process | DIY | Large (static) | Research, read-heavy, custom |
| Qdrant | OSS database | Self-host / cloud | Rich | Large | Balanced production |
| Weaviate | OSS database | Self-host / cloud | Rich + hybrid | Large | Hybrid search, modules |
| Milvus | OSS database | Self-host / cloud | Yes | Massive (distributed) | Billions of vectors |
| Pinecone | Managed | Cloud only | Yes | Large | Hands-off production |
| pgvector | Postgres extension | Wherever PG runs | SQL WHERE | Med | Already on Postgres |
| Redis Β· ES Β· Mongo | Existing DB + vectors | Various | Yes | Medβlarge | Already using them |
| If you⦠| Reach for⦠|
|---|---|
| Are prototyping locally with no infrastructure | Chroma or FAISS |
| Are already running PostgreSQL | pgvector |
| Want hands-off, managed production | Pinecone |
| Want self-hosted production with rich filtering | Qdrant or Weaviate |
| Need hybrid (vector + keyword) search built in | Weaviate (or Qdrant/ES) |
| Have billions of vectors / need distributed scale | Milvus |
| Already run Redis / Elasticsearch / Mongo | Their vector features |
| Question | Short answer |
|---|---|
| What does a vector DB do? | Store embeddings + metadata and find the nearest vectors to a query, fast. |
| Why not exact search? | Exact is O(NΒ·d) β too slow at scale; ANN indexes trade a little recall for big speed. |
| Common ANN index? | HNSW (a navigable graph); also IVF, PQ, DiskANN. |
| Library vs database vs managed? | FAISS = library; Chroma/Qdrant/Weaviate/Milvus = databases; Pinecone = managed; pgvector = Postgres. |
| Golden rule? | Same embedding model for indexing & querying; matched distance metric. |
| How to start? | Embedded (Chroma/FAISS) β graduate to a server when you need scale. |
Six questions on vector databases. Pick an answer for each, then check your score.
Where the vectors come from β how text becomes the vectors these databases store.
Retrieval in context β vector search is the engine behind RAG.
The full markdown guide β storing chunk data across Chroma, FAISS, Qdrant, Pinecone, pgvector.
The hub β LLMs, embeddings, retrieval, agents & more.