RAG vs Fine-Tuning vs Prompt Engineering: The LLM Customization Playbook

Your product needs a model that knows your business. Three routes promise that: write better prompts, retrieve your documents at query time, or retrain the model on your data. They are not three flavours of the same thing. They differ in cost, what they can teach, how fast they go stale, and what evidence they demand. Pick the wrong one and you pay the most expensive price in software: rebuilding the layer you should have chosen first.

This playbook is the cloud API build decision. It prices the three routes from live vendor pages read on 13 November 2026, runs the token math at a stated volume, and sets an eval gate. It is not the self-hosting decision; that deployment is priced in the companion guide linked at the end.

Who this playbook is for, and who should stop here

This is for a founder, product lead, or engineer shipping a customer-facing feature on a vendor API: support answers, document search, classification, extraction, drafting. You have real traffic, a monthly bill you can read, and someone accountable when the model invents a fact.

It is not for everyone. If you handle regulated data under a contract that forbids sending it to a vendor API, the cloud decision is already made for you: self-host or do not build it. If you need deterministic output, a template with a human check, or a rules engine, no amount of tuning will make a probabilistic model deterministic.

The three legs, defined by what they can teach

Prompt engineering changes the instructions, not the model. System prompt, few-shot examples, output schema, and the context you paste in. It teaches nothing permanent; it steers what the model already knows. OpenAI states the order plainly in its model optimization guide: evals first, then prompt engineering, then fine-tuning, repeating the loop. Prompt engineering is the step most teams under-invest in and the step that resolves most failures.

RAG, retrieval-augmented generation, changes the evidence, not the model. At query time the system searches your documents, retrieves the relevant passages, and hands them to the model with the question. Answers can cite sources, and the knowledge updates the moment the document updates.

Fine-tuning changes the weights. You upload a dataset of prompt-and-response pairs in JSONL, run a training job, and get a model that behaves in the way your examples show. What it teaches is format, tone, classification boundaries, and task behaviour. What it does not teach is new facts: a fine-tuned model has no more knowledge than its base model, and it will hallucinate in your house style.

The decision matrix

The price board, 13 November 2026

All prices are per 1M tokens, from the vendor pages opened this cycle. Google doubles its headline rates on 1 December 2026.

Two facts change decisions. First, fine-tuned gpt-4.1 costs three times the base-class input rate and more than twice the output rate of gpt-6.1-terra: fine-tuning the same tier is a price increase, not a saving. Second, budget on the December Gemini price, not the November one.

Worked example: 100,000 queries a month

Constructed example, stated inputs, verified prices. A support assistant answers 100,000 queries a month, 400 output tokens each. Prompt engineering uses an 800-token prompt; RAG adds 4,000 retrieved tokens per query and OpenAI file search.

Route A: prompt engineering on gpt-6.1-terra.

That is $0.0028 per query, the number every other route must beat.

Route B: RAG on the same model. Retrieved context is 4,000 tokens per query, so input becomes 4,800 tokens per query. OpenAI file search charges $2.50 per 1,000 tool calls and $0.10 per GB per day for storage, with 1 GB free.

RAG costs $957.00 a month against $280.00: 3.4 times as much, or $0.00957 per query. The file search calls at $250.00 are the part teams forget to budget. Pay the $677.00 premium only when grounded, citable answers carry refund or warranty consequences.

Route C: fine-tuning, priced honestly. First the trap: fine-tune gpt-4.1 with the same 800-token prompt at $3.00 input and $12.00 output:

That is $440.00 a month more than prompt engineering on the cheaper base model, before a cent of training. Fine-tuning the same tier loses. Now the case where it wins: substitution. The base model needed a 3,000-token few-shot prompt on gpt-6.1-sol at $2.00 input and $10.00 output, costing $600.00 input plus $400.00 output, or $1,000.00 a month. A fine-tuned gpt-4.1-mini takes the same task with an 800-token prompt at $0.80 input and $3.20 output:

Training a 2M-token dataset costs $10.00 per run at the $5.00 per 1M training rate. The saving is $808.00 a month, or $9,696.00 a year against about $40.00 of annual retraining. OpenAI says this in its own guide: train a smaller, cheaper, faster model to excel at a particular task where a larger model is not cost-effective. That sentence is the entire economic case for fine-tuning.

Two levers that cost nothing to build

Prompt caching. If your system prompt is stable, the vendor charges a fraction for re-reading it. Anthropic prices a 5-minute cache write at 1.25 times the base input rate and a 1-hour write at 2 times, with cache reads at 0.1 times base, and states caching pays after one read on the 5-minute tier and two reads on the 1-hour tier. Take Claude Sonnet 5.5 at $2.00 input, a 5,000-token system prompt, 100,000 queries a month. Uncached, that prompt costs $1,000.00. Cached with 1-hour writes: reads cost $50.00 and 720 writes cost $14.40, for $64.40 total, a saving of $935.60 a month. Apply caching before debating models.

Batch processing. Anthropic discounts batch API usage by 50 percent on input and output; Google discounts its batch tier by 50 percent. Anything that can wait minutes, classification backlogs, nightly extraction, bulk drafting, should run in batch. At this volume, batch pricing alone cuts the $280.00 route to $140.00.

The eval gate: measure before you choose, and after

OpenAI's optimization loop starts with evals and never leaves them: write evals, establish a baseline, change one thing, re-run. LangSmith frames the same discipline as two modes: offline evaluation, testing against a dataset before shipping, and online evaluation, monitoring live traffic as it happens.

Ragas supplies the metrics for the RAG side. Its index lists Context Precision, Context Recall, Context Entities Recall, Noise Sensitivity, Response Relevancy, and Faithfulness for retrieval work, plus Factual Correctness and Semantic Similarity for answer comparison. Faithfulness asks whether every claim in the answer is supported by the retrieved context. Context Precision asks whether the retrieved passages were the relevant ones. Together they separate a retrieval problem from a generation problem: low faithfulness with high precision means the model ignored good context; low precision means retrieval handed it the wrong documents.

This playbook gates shipment on three thresholds, measured on at least 200 real questions, not on a demo set you wrote to pass: faithfulness at least 0.90, context precision at least 0.80, response relevancy at least 0.85. Those numbers are this playbook's own gate, not vendor promises; no vendor publishes a passing score for your data. Below the gate, fix retrieval first: chunking, the embedding model, the number of passages retrieved.

The loop:

1. Write the eval set from real production questions, with expected answers.
2. Score the base model with a plain prompt and record the baseline.
3. Change one route and re-run the same set.
4. Ship only above the gate, and keep the failing questions as regression tests. For the companion deep dive on the metrics themselves, see the RAG evaluation metrics guide.

Data and eval requirements per route

The fine-tuning dataset is the hidden cost. Five hundred to a thousand labelled examples, each reviewed by a human who knows the right answer, is weeks of expert time. A dataset built by copying model outputs is a photocopy of the base model, not training data.

Failure modes: how each route breaks

Prompt engineering breaks quietly. A vendor updates the base model, your carefully tuned prompt drifts, and nobody notices for a month. Re-run the eval set on every model version change; a prompt is versioned code, not a note.

RAG breaks at the index. The corpus goes stale, a chunking change splits the one paragraph that answers the top question, and faithfulness falls while the model keeps answering fluently. The fix is operational: index freshness checks and the eval gate run on a schedule, not on launch day only.

Fine-tuning breaks at the data. Training examples leak test questions, the model learns the style of the training set instead of the task, or the base model is deprecated and the fine-tuned model dies with it. OpenAI's wind-down is the live example: a capability your roadmap assumed can be withdrawn by a pricing page update. Keep the prompt engineering fallback warm.

The build checklist

• [x] Eval set written: at least 200 real questions with expected answers.
• [x] Baseline measured: base model with a plain prompt.
• [x] Route chosen by the failing questions: retrieval failures mean RAG, format failures mean fine-tuning.
• [x] Monthly cost modelled: token math at real volume, from current vendor pricing.
• [x] Caching enabled: system prompt cached, batch API used for async work.
• [x] Gate thresholds set: faithfulness 0.90, precision 0.80, relevancy 0.85.
• [x] Fallback kept: prompt engineering version still works if the other routes fail.
• [x] Review date set: pricing re-checked quarterly.
• [x] Owner named: one person owns the eval set and the index.

The decision in advance

Start with prompt engineering and an eval set. Add RAG when answers must be grounded in documents that change, and price it at $957.00 a month at this volume before you promise it. Add fine-tuning only when it moves you to a smaller, cheaper model, and check first whether the vendor still sells it to you. For the other deployment shape, self-hosted models on your own hardware, work through the local LLM fine-tuning vs RAG guide. And if the arithmetic still feels abstract, run your own numbers with the break-even calculator.