What is a vector database?
A vector database stores the numerical representations produced by an embedding model and returns the entries closest to a query vector. Closeness is computed by a distance measure over an index designed for approximate search at scale. The approximation is the point, because an exact nearest-neighbour search over a large corpus is too slow to serve a question.
Also called: Vector store
Why the index you choose determines the recall you get
The vector database is the component teams select first and the one that explains least about retrieval quality. Almost any of them will return plausible neighbours on a small corpus, which is why a selection made during a pilot tells an organisation very little. What separates them in production is metadata filtering at scale alongside the recall an approximate index actually delivers under its configured parameters. Those properties stay invisible until the corpus is large enough to force a choice.
- Approximate indexes trade recall for speed, and the trade is configurable.
- Metadata filtering combined with vector search is where implementations differ most.
- Re-embedding a large corpus is a project rather than a configuration change.
- A pilot corpus is too small to reveal how an index behaves at scale.
How a vector database works
- Passages are encoded into vectors by the chosen embedding model.
- Vectors are stored with the metadata that filtering and entitlement checks require.
- An index structure is selected and its parameters are set against a recall target.
- A query is encoded by the same model and compared under the same distance measure.
- Candidate neighbours are returned with their distances and their metadata.
- Recall against exact search is measured whenever index parameters are altered.
How a vector database is evaluated
A vector database is assessed on behaviour at the corpus size it will actually hold. Measure the following under production conditions:
- recall of the approximate index against exact nearest-neighbour search
- query latency under concurrent load at the volume expected
- accuracy of filtered search compared with unfiltered search
- elapsed time and cost to re-embed and reindex the full corpus
- index size and memory required as the corpus grows
Limitations, risks and common failure modes
- Approximate index parameters are left at defaults, so recall is never known.
- Metadata filtering is applied after the vector search and silently empties the result set.
- A pilot corpus hides the latency the production corpus will produce.
- Changing the embedding model obliges a full re-embedding nobody scheduled.