Embeddings and vector search interview questions

Finding documents by meaning rather than exact keyword matches, typically using embeddings.

'car' can match 'automobile'.

A vector of numbers representing the meaning of text, so similar meanings are close together

Distance in embedding space approximates semantic similarity.

Cosine similarity

It compares direction regardless of length; for normalised vectors it equals the dot product.

Embeddings capture general meaning and may not preserve exact rare tokens like specific codes, so semantically similar but wrong documents can rank higher. Keyword or hybrid search matches the exact string reliably.

A common reason to add BM25 alongside vectors.

Combining keyword search (like BM25) with vector search and merging the results, to get both exact-term matching and semantic matching.

Helps with names, codes, and rare terms embeddings handle poorly.

Exact search over millions of vectors is too slow; ANN trades a little recall for big speed gains

Brute force is fine for thousands of vectors; ANN matters at scale.

A Postgres extension that adds a vector column type and similarity search, including ANN indexes.

Lets you keep vectors next to your relational data.

Build a small labelled retrieval eval from real queries and relevant documents, then compare candidate models on retrieval metrics like recall@k. Also weigh cost per token, latency, vector dimension (storage and speed), max input length, language support, and whether you can host it or depend on an API.

Public leaderboards are a starting point, not a decision.

Higher-dimensional embeddings can capture more nuance and sometimes improve retrieval quality, but they cost more storage and memory and make similarity search and indexing slower. Lower dimensions are cheaper and faster but may lose accuracy. Some models support truncating dimensions, so you can measure the quality loss on your eval.

It's a quality vs cost/speed decision you should measure.

Re-embed all documents with the new model

Different models produce incompatible vector spaces; queries and documents must use the same model.