RAG interview questions
Retrieval-augmented generation: chunking, retrieval, reranking, grounding. Ten practice questions with answers, for AI Engineer interviews.
Chunking
Show answer
Splitting documents into smaller pieces before embedding them, so retrieval can return focused, relevant passages.
Often by headings or paragraphs, with some overlap.
What problem does retrieval-augmented generation (RAG) mainly solve?
- Giving the model relevant, up-to-date or private information at query time
- Making the model faster
- Reducing the need for prompts
- Training the model on new data
Show answer
Giving the model relevant, up-to-date or private information at query time
RAG adds knowledge to the prompt; it doesn't change model weights.
List the main steps of a basic RAG pipeline from document to answer.
Show answer
Ingest documents, split them into chunks, embed the chunks and store them in a vector index (often with metadata). At query time, embed the query, retrieve the most similar chunks, optionally rerank, put them in the prompt with instructions to answer from them and cite sources, then generate the answer.
Offline indexing plus online retrieval and generation.
Contextual retrieval / chunk enrichment
Show answer
Adding document-level context (title, section, a short summary) to each chunk before embedding so isolated chunks are retrieved more accurately.
A chunk saying 'revenue grew 3%' is ambiguous without the company and quarter.
Grounding
Show answer
Having the model base its answer on supplied sources, ideally citing them, and say when the sources don't contain the answer.
Key to reducing hallucinations in RAG.
Chunks are too large. What's the most likely symptom?
- Retrieval returns chunks with lots of irrelevant text, diluting the answer and wasting context
- The embedding API fails
- Answers become more accurate
- No effect
Show answer
Retrieval returns chunks with lots of irrelevant text, diluting the answer and wasting context
Too small loses context; too large loses precision. Tune on your eval.
What does a reranker do in a RAG pipeline?
- Re-scores the top retrieved candidates with a more accurate (slower) model
- Generates the final answer
- Creates embeddings
- Splits documents into chunks
Show answer
Re-scores the top retrieved candidates with a more accurate (slower) model
Retrieve broadly and cheaply, then rerank precisely.
How do you evaluate a RAG system? Name what you'd measure for retrieval and for generation.
Show answer
Separately. For retrieval: on a labelled set of questions with known relevant chunks, measure recall@k / hit rate and ranking quality (e.g. MRR). For generation: measure answer correctness against references, faithfulness (is every claim supported by retrieved context), and citation accuracy, often with an LLM grader checked against human labels.
Separating them tells you which half to fix.
When is RAG the wrong solution, and what might you do instead?
Show answer
When the knowledge base is small enough to fit in the context window (just include it, with prompt caching); when the task needs a behaviour or style change rather than knowledge (prompting or fine-tuning); when the question needs aggregation over structured data (text-to-SQL or tools); or when exhaustive reasoning over a whole corpus is required, where retrieval of top-k chunks misses information.
Interviewers like candidates who know a tool's limits.
Users ask 'what changed in the policy since last year?' and RAG answers poorly. Which fix most directly helps?
- Add metadata (dates, versions) and filter or route retrieval by it
- Increase temperature
- Use bigger embeddings only
- Remove the system prompt
Show answer
Add metadata (dates, versions) and filter or route retrieval by it
Temporal and structured questions need metadata, not just semantic similarity.
Practise these with spaced repetition and AI grading, then try a mock interview.
Start Learning Free