Vector Search

Vector search retrieves records whose embeddings are close to a query embedding. It is commonly used when semantic similarity matters more than exact keyword overlap.

A system first creates embeddings for documents or items and stores them in a vector index. At query time it embeds the query and performs exact or approximate nearest-neighbor search using a chosen similarity metric.

Vector search is a core component of many semantic-search and RAG systems. Good retrieval still depends on chunking, embedding quality, metadata filtering, index configuration, and evaluation.

See also Embedding, Retrieval-Augmented Generation (RAG), and Latency and Throughput.

All document IDs here are fictional. Assume both searches use the same corpus, filters and distance metric. Exact top five results are {1, 2, 3, 4, 5}; approximate results are {1, 2, 3, 4, 9}. Four of the five exact neighbors were recovered, giving ANN recall@5 of 4 / 5 = 80%. This measures reproduction of exact nearest neighbors, not whether the documents answer the user's question.

If human-labeled relevant documents in the corpus are {2, 4, 9, 12, 15, 18}, the approximate results contain three relevant items: 2, 4 and 9. Evidence precision@5 is 3 / 5 = 60%, while evidence recall@5 is 3 / 6 = 50%. Record each metric's reference set and denominator. Exact vector geometry does not establish complete answer evidence.

In a fictional internal AI wiki, research-only and public documents may share an index, but retrieval must remain within the user's allowed corpus. Enforce authorization across retrieval, reranking and generation regardless of how an index implements filtering. Selecting five neighbors and removing forbidden items afterward may leave only two candidates. Evaluate candidate counts and search settings against the same authorized corpus without relaxing permissions to fill the list.

Distinguish insufficient allowed documents, incorrect filters and approximate-search misses. Sending forbidden passages to an agent tool result and hiding them only in the final answer does not replace access control at the retrieval boundary.

Hold source version, queries, filters and distance metric fixed, and compare the approximate index with exact search. Fictional timings of 30ms and 10ms are not product measurements. Without corpus size, dimension, hardware, concurrency and timing boundaries, they cannot establish a deployment advantage. Record p95 latency, ANN recall and human relevance metrics together.

Reranking reevaluates retrieved candidates. It cannot recover an answer document absent from that candidate set, so separate retrieval misses from ranking errors. Change chunking, candidate counts or reranking one at a time to make the comparison interpretable.

The FAISS paper ↗ investigates exact, approximate and compressed-domain similarity search on GPUs.^[1] The HNSW paper ↗ proposes hierarchical graph-based approximate nearest-neighbor search.^[3] Their dataset and hardware results are not guarantees for the current service.

For unexpected results, check source IDs and index versions, encoder consistency, dimensions, distance settings and permission filters. A new document present in source storage but absent from the index requires synchronization before a different retrieval model can help.

The HNSW paper ↗, algorithms 1, 2 and 5, distinguishes graph connections M, construction candidate list efConstruction and search candidate list ef.^[3] Final result count K is not the search breadth. Compare connection, construction and query costs separately from recall. Implementation names and constraints can differ; a tested setting is not a universal prescription across vector dimensions or distributions.

These are fictional measurements on the same questions, corpus, filters and k. Search p95 and ANN recall are A: 5ms/78%, B: 8ms/90%, C: 12ms/94%, D: 10ms/86%. B dominates D on these two metrics because it is faster with higher recall. A, B and C exchange speed for recall, so none is the unconditional best. Including memory, build time and update cost may change the comparison. Fix the latency measurement boundary and representation model.

This is an ORBiS Tree operational verification proposal. After source edits, deletion or permission changes, recompute exact neighbors over the same dated authorized corpus. Reusing old neighbors against a new index can misclassify a data change as an approximation failure. Preserve source IDs, corpus version, filters, tie handling and distance formula; distinguish cold-start from warmed-cache latency. Keep index lineage and check that deletions and permission changes propagate to answer evidence.