Skip to content
29Oct 2026

What Is RAG in AI? A Plain Guide for UK Organisations

RAG stands for retrieval-augmented generation. It is a way of making an AI model answer from your own approved documents: the system first looks up the relevant passages, then the model writes its answer using that material. That is why a RAG chatbot can quote your refund policy or membership rules instead of guessing.

This article explains how RAG works in plain terms, where it helps, where it goes wrong and which UK data-protection duties apply. It is written for owners, operations leads and membership managers who want AI that answers from controlled information, and who may not have technical staff.

TL;DR

  • RAG makes an AI model look up your approved documents before it answers, rather than relying on what it learned in training.
  • It is typically cheaper, faster to change and easier to check than retraining a model on your data.
  • Its answers are only as good as the documents behind it, so out-of-date or contradictory files produce poor answers.
  • UK GDPR duties still apply, so access rights, retention and supplier checks matter from day one.
  • Start with one narrow, well-documented use case, test it against real questions and keep a human in the loop for anything consequential.

What RAG is, in plain English

A large language model, the technology behind ChatGPT and similar tools, writes fluent text based on patterns learned in training. It does not know your pricing, your committee minutes or your member benefits. When asked about them, it may produce something plausible but wrong.

RAG fixes this by adding a lookup step. Think of an open-book exam. The model is not expected to remember the answer. It is handed the relevant pages and asked to answer from them.

The three words in the name describe the process:

  • Retrieval: the system searches your documents for passages that match the question.
  • Augmented: those passages are added to the instructions sent to the model.
  • Generation: the model writes a reply based on that material, ideally with a reference to the source.

Microsoft describes the same pattern in its Microsoft Learn overview of RAG, where a search index supplies the grounding data and the model produces the response. The idea is not tied to one vendor. It is a design pattern, and many business chatbots that answer from internal content use some form of it.

How RAG works, step by step

You do not need to build this yourself, but knowing the stages helps you ask suppliers sensible questions.

Preparing the content

Your documents, such as PDFs, web pages, policies, FAQs and help articles, are collected and cleaned. They are split into small chunks, often a few paragraphs each. Each chunk is converted into a numerical representation of its meaning, called an embedding, and stored in a searchable index.

Answering a question

  1. A person asks, "Can I transfer my membership to a colleague?"
  2. The system converts the question into the same kind of numerical representation.
  3. It finds the chunks closest in meaning, often combined with ordinary keyword search.
  4. It sends the question plus those chunks to the model, with instructions to answer only from them.
  5. The model replies and, if set up properly, shows which documents it used.

Many failures happen in steps one to three, not in the model. If the transfer rules are split across three documents, or the right chunk is never retrieved, the answer will be weak however clever the model is.

Pro Tip: Ask any supplier to show you the retrieved passages behind a wrong answer. If they cannot, you will struggle to fix problems later.

RAG versus fine-tuning and plain prompting

People often assume that to make AI know their business, they must train it on their data. That is rarely the right first step.

Plain prompting means pasting text into a chat. It works for one-off tasks, such as summarising a document, but does not scale to hundreds of files.

Fine-tuning adjusts the model itself using examples. It suits changes of style, format or specialist behaviour. It is a poor way to teach facts that change, because updating the knowledge means training again, and it is hard to trace an answer to a source.

RAG leaves the model alone and changes what it is shown. When your refund policy changes, you update the document and the next answer can reflect it. Answers can point back to a source, which makes checking far easier.

For many UK small and mid-sized organisations, RAG is a sensible starting point because it is easier to control, audit and correct. Fine-tuning can be added later if there is a clear reason. None of this removes the need for testing: RAG reduces invented answers but does not eliminate them.

Where RAG helps UK organisations

RAG suits any task where people repeatedly look things up in documents that already exist. Practical examples include:

  • Member and customer enquiries: a membership secretary fielding the same questions on renewal dates, fees, event terms and eligibility. A RAG chatbot answers from the approved handbook and passes the rest to a person.
  • Internal knowledge: staff asking where the expenses policy, onboarding checklist or supplier terms are. Our guide to a governance-first internal AI assistant covers how to set this up with permissions in place.
  • Support teams: agents drafting replies from the help centre and past approved answers, as described in our support pilot guide.
  • Tender and compliance responses: pulling accurate wording from previous submissions and policies for a person to review.

The common thread is a stable body of content, repeated questions and errors that a human can catch. Any time saved depends on how many enquiries are routine and how clean your documents are, so measure the current time spent before you set expectations with your board.

Data protection and security for RAG systems

RAG does not sit outside UK GDPR. If your documents contain personal data, such as member records, case notes or HR files, the rules apply to how they are indexed, retrieved and shown. The ICO's guidance on AI and data protection sets out what the law covers, including lawful basis, transparency, accuracy and data minimisation. It is guidance, not a compliance certificate, so take advice where your case is unusual.

Points to settle before launch:

  • Access control: the assistant should only retrieve what the person asking is allowed to see. A member should never be able to surface another member's record.
  • What goes in the index: leave out personal data you do not need. A policy library rarely needs names.
  • Supplier terms: check where data is processed, whether it is used to train models and what the contract says about retention.
  • Deletion: if a record must be erased, it must also leave the index and any copies.

On security, the NCSC's guidelines for secure AI system development are a useful reference. One RAG-specific risk is that a document in the index could contain hidden instructions that steer the model, so control who can add content.

Pro Tip: Carry the same permissions from your source system, such as SharePoint or your membership platform, into the index. Do not rebuild access rules by hand.

Where RAG goes wrong, and where to leave it out

The most common cause of poor answers is poor source material. Old policies sitting beside new ones, duplicate files and documents that contradict each other all confuse retrieval. A tidy-up of the content often does more than any technical tweak.

Other typical problems:

  • Wrong passage retrieved: the answer is fluent but based on the wrong section.
  • Partial answers: the rule has exceptions in another document that were never retrieved.
  • Confident gaps: the model fills a missing detail rather than saying it does not know.
  • Stale content: nobody owns updating the documents, so the assistant drifts out of date.

Be careful about where you use it. Do not let a RAG system give final decisions on disciplinary matters, medical or legal advice, complaints outcomes or anything that changes a person's rights. Use it to draft or point to sources, with a person responsible for the result.

It also cannot answer from information you have not written down. If the real rule lives in someone's head, the first job is documenting it. For the next step up, where a system takes actions as well as answering, see our plain guide to what an AI agent is.

RAG in tools you may already have

You may already be using RAG without calling it that. Microsoft 365 Copilot answers from emails, files and meetings that you have permission to see, which is retrieval and generation working together. Many help-desk and website chat products offer an option to answer from your help centre.

Off-the-shelf tools are often the right answer for generic tasks. A custom build makes sense when your content sits across several systems, such as a CRM, a membership database, a website and finance software, or when you need tight control over sources, tone and handover.

RAG is also relevant to how your own content is found. Search engines now generate answers from web pages too, and Google explains how its AI features draw on site content in its Search Central documentation. Clear, accurate, well-structured pages help both your own assistant and public AI search, though nothing guarantees a particular result.

How to start a RAG project sensibly

Keep the first project small and measurable. A sensible sequence looks like this:

  1. Pick one use case. For example, the twenty most common member questions.
  2. Audit the content. Remove duplicates, retire outdated files and name an owner for each document.
  3. Write a test set. Collect 50 to 100 real questions with correct answers agreed by staff.
  4. Build and measure. Check accuracy, whether the right source was cited and how often it correctly says it does not know.
  5. Pilot with a small group. Keep a visible route to a human and log what people ask.
  6. Review monthly. Fix gaps in the documents first, then adjust the system.

Complete a data protection impact assessment where the ICO guidance suggests one is needed, and tell people when they are talking to AI. The government's broader approach to AI is set out by the Department for Science, Innovation and Technology, which is worth following as guidance evolves.

Pro Tip: Treat the "I don't know" rate as a success measure. An assistant that declines to answer when the documents are silent is safer than one that always replies.

After more than 20 years of building business systems, I have seen the same pattern repeatedly: the technology is rarely the problem, the information underneath it is. RAG works well when your documents are current and owned by someone. It works badly when they are a pile of old PDFs. I would never let it make a final call on a member's complaint or a disciplinary matter. Use it to find and draft, and keep a person accountable for what goes out.

- Rob, Cloud 9

About Cloud 9

Cloud 9 helps UK businesses, membership organisations and trade associations put AI to work where it genuinely pays off. We start with AI strategy and readiness to check whether your content and systems are fit for RAG, then build AI agents and chatbots that answer from your approved information, connected to your CRM, membership platform and finance software. We will also tell you when AI is the wrong tool. To talk through a use case, book a discovery call.

Frequently asked questions

Is RAG the same as training an AI on my data?

No. Training or fine-tuning changes the model itself. RAG leaves the model unchanged and gives it relevant passages from your documents at the moment of each question. This makes it easier to update and easier to trace to a source, and it is often quicker and cheaper to set up for business knowledge.

Does RAG stop AI making things up?

It reduces the problem but does not remove it. The model can still misread a passage, miss an exception or fill a gap. Good retrieval, clean documents, source citations, testing against real questions and a clear "I don't know" behaviour all help. Human review stays essential for important answers.

Is RAG allowed under UK GDPR?

UK GDPR does not ban RAG, but it applies if personal data is in the documents or questions. The rules cover areas such as lawful basis, security, supplier arrangements and retention. The ICO publishes guidance on AI and data protection, and you should seek advice for your specific circumstances.

Do I need a developer to use RAG?

Not always. Some off-the-shelf tools offer RAG-style answers from your files or help centre with little setup. If your information sits across several systems, or you need access controls and handover to staff, you will usually need a specialist to integrate and test it properly.

Sources

  1. Retrieval augmented generation (RAG) in Azure AI Search, Microsoft Learn
  2. Guidance on AI and data protection, ICO
  3. Guidelines for secure AI system development, NCSC
  4. AI features and your website, Google Search Central
  5. Department for Science, Innovation and Technology, GOV.UK