The term RAG is broken

RAG stands for Retrieval Augmented Generation. The idea is elegant: before answering, the model queries a knowledge base and uses that context to generate the answer. On paper it sounds infallible. In production there are two implementations that share the name and behave differently: generic RAG and strict RAG.

Generic RAG

The default architecture. The corpus is embedded into a vector store, semantic search runs, the top-k chunks enter the prompt, and the model answers with that information as a suggestion. The problem: the model feels free to complete the answer with its general knowledge whenever it “believes” that helps.

That trait makes it fascinating for marketing chatbots and dangerous for regulated operations. It states true things with the same cadence it states invented ones. When a credit union advisor quotes a rate or a lawyer quotes a ruling, the difference between truth and hallucination stops being aesthetic: it is legal.

Strict RAG

The architecture we use when the answer lives under a norm. Three non-negotiable rules:

Mandatory citation. Every factual claim must be traceable to the chunk that supported it. With no source, the agent says “I did not find the information”.
No completion with general knowledge. The model does not add what it believes it knows. If the corpus does not cover it, the case is escalated.
Exportable traceability. Every answer is signed into the audit ledger with the list of chunks consulted and the hash of the source document.

When each one makes sense

Generic RAG works well for discovery and first line support where the right answer is “as close as possible”. It is fast to implement and conversational. The bar: the cost of an error is low and the user tolerates the agent recommending they verify with a human.

Strict RAG becomes mandatory in tax law, financial compliance, regulated product, health and any domain where the end customer needs to defend itself before an auditor. It is not an optional upgrade: it is a change of contract with the model.

What it costs to switch

Strict RAG demands more chunking work, more metadata in the corpus, continuous evaluations and an agent that can say “I do not know” as fluently as it answers. The investment pays for itself the first case that clears an audit without a single data point unsupported by its source.

The field rule is direct: if your customer may one day have to prove where an answer came from, choose strict RAG from day zero. Converting a generic RAG into a strict one in production costs more than designing it right.