What Is RAG, & When Does Your Business Need It?

RAG (retrieval augmented generation) is a pattern where software finds relevant passages from your own documents first, then asks a model to answer using those passages as evidence. You need it when accuracy against private material matters more than fluent guessing - contracts, manuals, diligence packs, product specs - and when you can invest in a corpus, evaluation and release gates. Fine-tuning a model on the same text is a different tool for a different problem; most business “chat with our PDFs” jobs are RAG jobs.

RAG in one paragraph

What is RAG in AI in simple terms?

RAG in AI means: search your knowledge base for the chunks that matter, stuff those chunks into the prompt, generate an answer, and preferably show citations so a human can check the source. The model is not “remembering your company” from training. It is reading retrieved text at answer time. That is why fresh documents can appear without retraining the whole model, and why garbage in the corpus still becomes garbage in the reply.

Components in practice:

• A document store (files, pages, tickets, paragraphs)
• Chunking and embeddings so similar meaning is findable
• A retriever that ranks the best chunks for a query
• A generator (usually an LLM API) that drafts from those chunks
• Logging, evaluation and a human path when confidence is low

Service context: AI development. Sibling definitions: agentic AI for multi-step action systems, and AI development company for who builds this class of product.

Why chatbots hallucinate and how retrieval fixes it

A naked chatbot answers from pattern completion. Asked about your 2024 supplier schedule, it may invent a confident paragraph because fluency is cheap and truth is not in the weights. Users experience that as “AI lying”. Engineers experience it as missing grounding.

Retrieval fixes the common case by narrowing what the model is allowed to lean on. You still get hallucinations when:

• The wrong chunks are retrieved
• Chunks contradict each other
• The prompt lets the model fill gaps instead of saying “not in sources”
• Nobody evaluates answers against a golden set

So RAG is not magic honesty. It is an architecture that makes honesty possible when your corpus and evaluation are real. Monitoring drift and bad retrieval matters as much as picking a model brand.

[ retrieval, not memory ] The model doesn't know your documents. It reads them, one chunk at a time

01 CONTRACT 02 INVOICE 03 MANUAL 04 EMAIL 05 SPEC SHEET 06 TICKET

READING THE SOURCES

BUILD BOT_ CHUNKS READ

SIX CHUNKS LATER ONE CITED ANSWER Six passages checked, one answer grounded in the text

RAG vs fine-tuning vs bigger prompts: the decision that actually matters

What is the difference between RAG and fine-tuning?

RAG pulls evidence at query time from an external store. Fine-tuning updates model weights on examples so behaviour or style shifts. Bigger prompts (and long context windows) stuff more text into a single call without a separate retrieval index.

Approach	Best when	Weak when
Bigger prompts / long context	Small, stable doc set; simple demos	Large corpora; cost per call explodes; weak citations
RAG	Private, changing knowledge; need citations	Corpus is empty, messy or legally untouchable
Fine-tuning	Style, format, specialised behaviour on stable tasks	You mainly need up-to-date facts from internal docs

"…He has taken the time to understand both the commercial and technical requirements, communicated clearly throughout, and delivered a well-thought-out system with a high level of flexibility. The admin dashboard, in particular, gives us full control over the scoring and valuation logic, which was a key requirement…"

When should a business use RAG?

Use RAG when staff already lose hours searching SharePoint, drives or PDFs; when answers must point at a clause or page; and when the document set changes faster than you would retrain. Skip RAG when the job is creative drafting with no source of truth, when you only need a public FAQ the model already knows, or when you refuse to clean the corpus. Also skip unsupervised customer-facing RAG on high-stakes advice until review gates exist - retrieval reduces risk, it does not remove accountability.

Rule of thumb from our builds: if the user should be able to click a citation, start with RAG. If the user needs the model to write in your brand voice on open tasks, consider light fine-tuning or strong system prompts after grounding is solved.

What we built: diligence-grade document comparison

We shipped a UK document-comparison platform - diligence-grade RAG over private materials using the Claude API. The point of the product is judgement with evidence - not a novelty chat window on a marketing site.

Architecture in plain terms:

• Ingest and chunk virtual-data-room-style documents
• Retrieve the passages relevant to a compare or question
• Generate structured comparison output with source grounding
• Keep humans in release authority for high-stakes interpretation
• Log enough to debug bad retrieval and improve the evaluation set

Outcomes that matter to buyers of this pattern: faster first-pass compare, fewer “where did that claim come from?” loops, and a system you can harden rather than a demo that dies after the workshop. The durable claim is production RAG against private docs with citations and human gates, which is the bar most “AI search” pitches miss.

Same studio pattern shows up elsewhere: agentic delivery on implementation, seniors on architecture, evaluation before Launch. Related commercial framing: AI consultant cost when you are still shaping the problem.

What a RAG project costs and how long it takes

How much does a RAG system cost to build?

In the UK, a thin RAG proof on a tidy corpus often sits around £15,000-£40,000 in the wider market; a production assistant with auth, citations, evaluation and monitoring commonly lands £40,000-£120,000+; multi-collection or regulated deployments run higher when permissions, redaction and audit trails dominate. Those are market ranges across agencies and product shops - not a Code23 quote. Corpus cleanup, connector work and evaluation design move the number more than the logo on the model API.

Slice	Typical UK market range	What you are buying
Proof / pilot	~£15k-£40k	One collection, thin UI, first evaluation set
Production assistant	~£40k-£120k+	Auth, citations, logging, review gates, handover
Complex / regulated	£120k+	Multiple stores, strict permissions, audit, change control

Running costs after launch include model API usage, hosting, embedding refresh, and people time for corpus hygiene. Anything outside the agreed scope is priced separately before we do it; continuous engineering and support can sit on support tiers when you want us watching the boring parts.

Timeline orientation: a focused pilot can land in weeks when documents and owners are ready; production Harden tracks permission matrices and evaluation quality. Agent-assisted implementation compresses coding; it does not invent a clean corpus. Method notes: build-time data.

Bring a sample document set and the five questions staff already ask. Diagnose RAG versus better search versus a process change through AI development.

Frequently asked questions

Does ‘RAG’ always mean AI in business conversations?

No. In a lot of UK businesses, RAG means red, amber, green: a traffic-light rating on a project status report, nothing to do with AI. This article is about the other RAG, retrieval augmented generation, the pattern that lets an AI system answer from your own documents with citations. Worth checking which one a colleague means before you start explaining embeddings.

Is RAG the same as a chatbot?

No. A chatbot is the interface, the window someone types into. RAG is the plumbing behind it: search your documents, retrieve the right passages, then generate an answer grounded in them. You can bolt a chatbot onto RAG, put the same retrieval behind a plain search box, or wire it into a workflow with no chat window at all. The interface is the easy part.

Can RAG keep up with documents that change every day?

Better than fine-tuning can. RAG reads your live document store at answer time, so a new contract or an updated policy is available as soon as it’s indexed, with no retraining needed. The corpus itself still needs upkeep: outdated or duplicate documents get retrieved and quoted just as confidently as good ones, so someone has to own keeping the source material current.

Do I need my own AI team to build a RAG system?

You don’t need an in-house AI team. A development partner can handle ingestion, retrieval and evaluation while your team owns the corpus, access rules and approval decisions.

Next step

If RAG sounds like the right shape for your problem, the way to find out is with a real document set and the five questions your team already asks it. We’ll tell you plainly whether it’s RAG, better search, or a process fix, before you spend anything on a build.

We’ve helped 50+ businesses implement AI. If accuracy against your own material matters more than a fluent guess, get in touch.