When a base model doesn't do quite what you need, you have three levers to adapt it β prompting, RAG, and fine-tuning. They solve different problems, and the classic mistake is fine-tuning when you needed retrieval, or vice-versa. This is the decision guide.
The one-liner. Prompting changes how you ask. RAG changes what the model knows right now (facts). Fine-tuning changes how the model behaves (skill, style, format). Reach for them in that order of effort.
Framing The three levers
Cheapest Β· instant
βοΈ Prompting
Craft the input β instructions, examples, format. No training, no infra.
Fresh Β· factual
π RAG
Retrieve relevant data at query time and put it in the prompt. Knowledge without retraining.
Behavior Β· style
ποΈ Fine-tuning
Train the model's weights on examples to change how it responds.
Key distinction: for knowledge ("what does it know?") use RAG; for behavior ("how does it act / what format / what style?") use fine-tuning. Fine-tuning is a poor way to inject facts β they go stale and it can hallucinate.
Lever 1 Prompting
βοΈ Prompting
Great for Steering tone, format and simple behavior; the first thing to try for almost everything. Zero training, instant iteration, works on any model. See prompt & context engineering.
Limits Bounded by the context window; can't add lots of private knowledge permanently; long few-shot prompts cost tokens every call.
Lever 2 RAG (retrieval-augmented generation)
π RAG
Great for Grounding answers in your data and current facts β docs, tickets, a knowledge base β that change often. Update the data, not the model; cite sources; scales to huge corpora. See RAG vs CAG.
Limits Answer quality is capped by retrieval quality; adds infra (embeddings + vector DB) and per-query latency; doesn't change the model's skills or style.
Lever 3 Fine-tuning
ποΈ Fine-tuning
Great for Teaching a durable behavior: a consistent style/voice, a niche format or task, a domain's language, or shrinking a big prompt into the weights (lower per-call cost/latency). Modern LoRA/PEFT tunes a small adapter cheaply rather than the whole model.
Limits Needs a quality labeled dataset (often hundredsβthousands of examples), a training pipeline, and eval; facts baked in go stale and can't be updated without retraining; risk of overfitting or degrading general ability. Highest effort of the three.
Common myth: "fine-tune it on our docs so it knows them." That's usually the wrong tool β use RAG for knowledge. Fine-tune for how it responds, not what it knows.
Choose How to decide
Prompting
RAG
Fine-tuning
Changes
How you ask
What it knows now
How it behaves
Effort / cost
Lowest
Medium (infra)
Highest (data + training)
Freshness
β
Live (update data)
Stale (retrain to update)
Iteration speed
Instant
Fast
Slow
Best for
Style, simple tasks
Private/current facts
Consistent skill, style, format
Not either/or Combining them
These stack, and mature systems use all three: a fine-tuned model for the right behaviour/format, fed retrieved facts (RAG) for grounding, driven by a well-engineered prompt.
ποΈ fine-tuned behaviour+π retrieved facts+βοΈ good prompt
Order of adoption. Exhaust prompting first (cheapest), add RAG when you need private/fresh knowledge, and only fine-tune once you've proven prompting+RAG can't get the behaviour you need β and you have the data and evals to do it well.
Cheat sheet
Question
Short answer
Each changes what?
Prompt = how you ask Β· RAG = what it knows now Β· Fine-tune = how it behaves.
Inject knowledge β ?
RAG (not fine-tuning β facts go stale).
Enforce a style/format β ?
Prompting first; fine-tune if it must be rock-solid.
Cheapest / fastest to try?
Prompting.
Cheap fine-tuning?
LoRA / PEFT (train a small adapter, not the whole model).
Best real-world answer?
Often all three combined.
Knowledge check
Six questions. Pick an answer for each, then check your score.