RAG

Prompting for Retrieval-Augmented Generation

RAG works best when the prompt clearly tells the model how to use retrieved context and what to do when context is incomplete.

Core rule

Tell the model to answer from retrieved material first. Also tell it how to behave when the context is missing, conflicting, or insufficient.

You are a grounded assistant.

Use only the context below to answer the question.
If the answer is not fully supported by the context, say so clearly.
Do not invent missing facts.

Context:
{retrieved_documents}

Question:
{user_question}

Output:
- direct answer
- supporting evidence from context
- uncertainty or missing information

Common RAG mistakes

Mistake	What happens	Fix
Too much retrieved text	The answer becomes noisy or confused.	Reduce chunk count and improve retrieval quality.
No grounding rule	The model mixes context with guesswork.	Explicitly say use only the context.
No uncertainty handling	The answer sounds certain even when evidence is weak.	Require the model to state gaps.