What Is RAG, & When Does Your Business Need It?
Retrieval augmented generation in plain English: what RAG is, when it beats fine-tuning, what it costs to build, and a real UK RAG project we shipped.
RAG (retrieval augmented generation) is a pattern where software finds relevant passages from your own documents first, then asks a model to answer using those passages as evidence. You need it when accuracy against private material matters more than fluent guessing - contracts, manuals, diligence packs, product specs - and when you can invest in a corpus, evaluation and release gates. Fine-tuning a model on the same text is a different tool for a different problem; most business “chat with our PDFs” jobs are RAG jobs.
RAG in one paragraph
What is RAG in AI in simple terms?
RAG in AI means: search your knowledge base for the chunks that matter, stuff those chunks into the prompt, generate an answer, and preferably show citations so a human can check the source. The model is not “remembering your company” from training. It is reading retrieved text at answer time. That is why fresh documents can appear without retraining the whole model, and why garbage in the corpus still becomes garbage in the reply.
Components in practice:
- A document store (files, pages, tickets, paragraphs)
- Chunking and embeddings so similar meaning is findable
- A retriever that ranks the best chunks for a query
- A generator (usually an LLM API) that drafts from those chunks
- Logging, evaluation and a human path when confidence is low
Service context: AI development. Sibling definitions: agentic AI for multi-step action systems, and AI development company for who builds this class of product.
Why chatbots hallucinate and how retrieval fixes it
A naked chatbot answers from pattern completion. Asked about your 2024 supplier schedule, it may invent a confident paragraph because fluency is cheap and truth is not in the weights. Users experience that as “AI lying”. Engineers experience it as missing grounding.
Retrieval fixes the common case by narrowing what the model is allowed to lean on. You still get hallucinations when:
- The wrong chunks are retrieved
- Chunks contradict each other
- The prompt lets the model fill gaps instead of saying “not in sources”
- Nobody evaluates answers against a golden set
So RAG is not magic honesty. It is an architecture that makes honesty possible when your corpus and evaluation are real. Monitoring drift and bad retrieval matters as much as picking a model brand.
RAG vs fine-tuning vs bigger prompts: the decision that actually matters
What is the difference between RAG and fine-tuning?
RAG pulls evidence at query time from an external store. Fine-tuning updates model weights on examples so behaviour or style shifts. Bigger prompts (and long context windows) stuff more text into a single call without a separate retrieval index.
| Approach | Best when | Weak when |
|---|---|---|
| Bigger prompts / long context | Small, stable doc set; simple demos | Large corpora; cost per call explodes; weak citations |
| RAG | Private, changing knowledge; need citations | Corpus is empty, messy or legally untouchable |
| Fine-tuning | Style, format, specialised behaviour on stable tasks | You mainly need up-to-date facts from internal docs |
When should a business use RAG?
Use RAG when staff already lose hours searching SharePoint, drives or PDFs; when answers must point at a clause or page; and when the document set changes faster than you would retrain. Skip RAG when the job is creative drafting with no source of truth, when you only need a public FAQ the model already knows, or when you refuse to clean the corpus. Also skip unsupervised customer-facing RAG on high-stakes advice until review gates exist - retrieval reduces risk, it does not remove accountability.
Rule of thumb from our builds: if the user should be able to click a citation, start with RAG. If the user needs the model to write in your brand voice on open tasks, consider light fine-tuning or strong system prompts after grounding is solved.
What we built: diligence-grade document comparison
We shipped a UK document-comparison platform - diligence-grade RAG over private materials using the Claude API. The point of the product is judgement with evidence - not a novelty chat window on a marketing site.
Architecture in plain terms:
- Ingest and chunk virtual-data-room-style documents
- Retrieve the passages relevant to a compare or question
- Generate structured comparison output with source grounding
- Keep humans in release authority for high-stakes interpretation
- Log enough to debug bad retrieval and improve the evaluation set
Outcomes that matter to buyers of this pattern: faster first-pass compare, fewer “where did that claim come from?” loops, and a system you can harden rather than a demo that dies after the workshop. The durable claim is production RAG against private docs with citations and human gates, which is the bar most “AI search” pitches miss.
Same studio pattern shows up elsewhere: agentic delivery on implementation, seniors on architecture, evaluation before Launch. Related commercial framing: AI consultant cost when you are still shaping the problem.
What a RAG project costs and how long it takes
How much does a RAG system cost to build?
In the UK, a thin RAG proof on a tidy corpus often sits around £15,000-£40,000 in the wider market; a production assistant with auth, citations, evaluation and monitoring commonly lands £40,000-£120,000+; multi-collection or regulated deployments run higher when permissions, redaction and audit trails dominate. Those are market ranges across agencies and product shops - not a Code23 quote. Corpus cleanup, connector work and evaluation design move the number more than the logo on the model API.
| Slice | Typical UK market range | What you are buying |
|---|---|---|
| Proof / pilot | ~£15k-£40k | One collection, thin UI, first evaluation set |
| Production assistant | ~£40k-£120k+ | Auth, citations, logging, review gates, handover |
| Complex / regulated | £120k+ | Multiple stores, strict permissions, audit, change control |
Running costs after launch include model API usage, hosting, embedding refresh, and people time for corpus hygiene. Our extras rate on scoped work is transparent fixed change bands; continuous engineering and support can sit on support tiers when you want us watching the boring parts.
Timeline orientation: a focused pilot can land in weeks when documents and owners are ready; production Harden tracks permission matrices and evaluation quality. Agent-assisted implementation compresses coding; it does not invent a clean corpus. Method notes: build-time data.
Bring a sample document set and the five questions staff already ask. Diagnose RAG versus better search versus a process change through AI development.
