LLM APIs and tool use interview questions
Messages APIs, streaming, and letting models call your functions. Ten practice questions with answers, for AI Engineer interviews.
Stop reason
Show answer
A field in the API response explaining why generation ended, such as end of turn, max tokens reached, a tool call, or a refusal.
Always check it: e.g. 'max_tokens' means the output is truncated.
In a typical chat-style LLM API, how does the model 'remember' earlier turns?
- You resend the conversation history with each request
- The server stores it automatically forever
- The model's weights update after each turn
- It doesn't; each answer ignores history
Show answer
You resend the conversation history with each request
Messages APIs are stateless: memory is whatever you include in the request.
What is 'tool use' (function calling)?
- The model outputs a structured request to call a function you defined; your code runs it and returns the result
- The model executes arbitrary code on the provider's servers by itself
- A way to fine-tune a model
- A billing feature
Show answer
The model outputs a structured request to call a function you defined; your code runs it and returns the result
The model decides what to call with which arguments; your application does the execution.
Why should LLM API calls be made from your server rather than directly from the browser?
Show answer
Calling from the browser would expose your API key to anyone, letting them run up your bill. A server route lets you keep the key secret, enforce auth, rate limits, and usage quotas, validate inputs and outputs, and log usage.
API keys in client bundles are a classic security mistake.
Tool schema
Show answer
A name, description, and JSON Schema for a function's input that you give the model so it knows when and how to call it.
Good descriptions matter as much as good prompts.
Why use streaming responses in a chat UI?
- Users see text as it's generated, which cuts perceived latency
- Streaming makes responses cheaper
- Streaming makes answers more accurate
- It's required for tool use
Show answer
Users see text as it's generated, which cuts perceived latency
Total time is similar, but time-to-first-token drops dramatically.
Exponential backoff
Show answer
Retrying a failed request after waiting progressively longer intervals (often with random jitter), used for rate limits and transient errors.
Prevents hammering an overloaded API and synchronised retry storms.
Describe the request/response loop for a model that uses a weather tool to answer 'Do I need an umbrella in Baltimore?'
Show answer
You send the user message plus the weather tool definition. The model responds with a tool call (e.g. get_weather with location Baltimore) and stops. Your code calls the real weather API, then sends a new request with the full history including the model's tool call and a tool result containing the forecast. The model then writes the final answer using that data.
The model never calls the API itself; your application is in the loop.
How would you make tool use safe when the tools can take real actions (send email, issue refunds)?
Show answer
Apply least privilege (only expose needed tools with narrow parameters), validate and authorise every call server-side as the actual user, require human confirmation for irreversible or high-impact actions, set limits (amounts, rate, recipients), log everything for audit, and treat any content the model read from external sources as untrusted so injected instructions can't trigger actions.
The model's request is a suggestion; your code enforces policy.
A tool call fails because an external API is down. What should you usually send back to the model?
- A tool result describing the error, so the model can retry, choose another approach, or tell the user
- Nothing; end the conversation
- A fake successful result
- The raw stack trace in a system prompt
Show answer
A tool result describing the error, so the model can retry, choose another approach, or tell the user
Models handle errors well when told plainly. Faking success leads to confidently wrong answers.
Practise these with spaced repetition and AI grading, then try a mock interview.
Start Learning Free