Retrieval-augmented generation, usually shortened to RAG, gives a language model selected information at the moment it answers. Instead of relying only on patterns learned during training, an application searches a document collection, adds relevant passages to the request, and asks the model to answer from that evidence.
The basic flow has three parts. Retrieval finds candidate passages. Augmentation packages those passages with instructions and the user’s question. Generation produces an answer that should remain grounded in the supplied material. This pattern is useful when knowledge is private, changes frequently, or needs citations.
RAG does not make a model inherently truthful. It creates a chain of components that can be inspected and improved. The retriever may miss the right document, select an outdated version, or return a passage that shares keywords but not meaning. The prompt may fail to require grounding. The model may then combine valid excerpts into an unsupported conclusion.
Document preparation determines much of the result. Files need stable identifiers, useful metadata, access rules, and chunks that preserve enough surrounding meaning. Chunks that are too large waste context and blur relevance. Chunks that are too small separate a claim from its definition, date, exception, or table heading.
Retrieval can use keyword search, vector similarity, semantic ranking, or a hybrid. Keyword search is strong for names, codes, and exact phrases. Vector search can find conceptually related passages even when wording differs. Hybrid retrieval is often a better default because it combines both signals, but it still needs evaluation against real questions.
Access control belongs in retrieval, not in a disclaimer after generation. A user should never receive a passage they were not allowed to retrieve. Retrieved documents must also be treated as untrusted input: a document can contain instructions that attempt to override the application’s rules. Keep system policy separate and restrict what downstream tools can do.
The prompt should identify every source chunk, require citations for factual claims, and define the fallback when evidence is insufficient. It should also tell the model how to handle disagreement: present the conflicting versions with dates and citations instead of silently choosing one. These requirements make errors easier to diagnose.
Measure retrieval and generation separately. Retrieval tests ask whether the supporting passage appeared among the top results. Generation tests ask whether the answer is supported, complete, correctly cited, and appropriately uncertain. A polished answer can hide weak retrieval, while perfect retrieval can still be followed by a bad synthesis.
More context is not always better. Low-relevance passages consume tokens and can pull the answer away from the strongest evidence. Set a relevance threshold, remove duplicates, prefer current versions when freshness matters, and reserve enough space for the answer. Complex questions may need to be decomposed into several focused searches.
RAG is the right tool when facts live in a changing or private corpus. Fine-tuning is usually a better fit when the goal is to change behavior, style, or task performance rather than inject current knowledge. Agent tools are useful when the system must decide when and where to retrieve as part of a larger workflow.
A trustworthy RAG interface should show its sources, admit when the corpus cannot answer, and make corrections possible. The real product is not the generated paragraph. It is the evidence path that lets a reader decide whether that paragraph deserves confidence.
AI Tools Radar separates product facts, editorial judgment, and commercial placement. Updated facts retain their verification date.