Three techniques get proposed for making an AI system fit your business, and they are routinely confused with one another. They solve different problems, cost different amounts, and there is a sensible order for reaching for them.
Prompting
What it is: writing better instructions — describing the task, audience, format and constraints, and supplying examples of good output.
What it fixes: outputs in the wrong tone, wrong format, wrong length, or missing something you needed. In other words, most initial disappointment.
Cost: hours, no infrastructure, immediately reversible.
This is where to start, always. A significant share of problems that get proposed as justifications for fine-tuning turn out to be a prompt that never specified what was wanted.
Retrieval (RAG)
What it is: finding relevant passages from your documents and supplying them with the question, so answers are grounded in your material.
What it fixes: the system not knowing your specific facts — your policies, products, contracts, procedures. Also fixes staleness, since updating a document updates the answers.
Cost: moderate. Document processing, an index, retrieval logic, and ongoing attention to keeping content current.
Reach for this when the gap is knowledge. If the complaint is "it doesn't know our return policy", this is the answer — not fine-tuning.
Fine-tuning
What it is: further training of a model on many examples of your inputs and desired outputs, adjusting the model itself.
What it fixes: consistent adherence to a very specific style or output format across large volumes, and sometimes reduced cost per request because instructions need not be repeated.
Cost: high. You need a substantial set of high-quality examples — typically hundreds to thousands — plus the training run, evaluation, and a repeat of all of it when requirements change or you want to move to a newer base model.
Critically: fine-tuning is a poor way to teach facts. It shapes behaviour and style. Facts taught this way are absorbed unreliably, cannot be cited, and are awkward to update. If someone proposes fine-tuning so the model "knows your data", that is a signal to ask more questions.
The order
- Fix the prompt. Add examples. Measure on real cases.
- If the gap is missing knowledge, add retrieval.
- If, after both, you still need rigid stylistic consistency at volume — consider fine-tuning.
Most companies never reach step three, and that is a good outcome rather than a shortfall.
Diagnosing which you need
Ask what is actually wrong with the output.
Wrong facts about your business → retrieval. Right facts, wrong format or tone → prompting. Correct and well-formatted but inconsistent across thousands of items despite a good prompt → possibly fine-tuning. Confidently invented specifics → retrieval plus an explicit instruction that it may decline to answer.
A note on cost
These are often presented as alternatives on a capability ladder, which encourages reaching for the most elaborate. In practice the ladder is one of cost and commitment. Prompting is nearly free and fully reversible. Fine-tuning creates an asset you must maintain, and ties you to decisions that get expensive to revisit.
Start cheap and reversible. Escalate only when you have evidence, from real cases, that the cheaper option cannot get there.
