Reverse RAG: Reduce Hallucinations and Errors in Medical GenAI — Part 1 – Usman's blog

Mayo Clinic adopted a reverse RAG technique that effectively eliminated data retrieval hallucinations in their tests. In a traditional RAG setup, an LLM retrieves context from a knowledge source before generating an answer. Mayo’s reverse RAG flips this process: the model first extracts or summarizes information, then links every data point in its output back to the document. By forcing the AI to provide a reference for each fact, Mayo virtually eliminated hallucinations in non-diagnostic use cases, building clinician trust in the results.

The workflow looks like this:

1. Data Extraction — The LLM/OCR/API reads the patient’s records (e.g. discharge summaries or outside medical files) and produces a summary or list of facts. This initial output might include details as patient age, diagnoses, lab results, etc.
2. Fact Splitting — The AI output is split into individual facts or data points. Each sentence or key piece of information from the summary is treated separately.
3. Source Matching — For each fact, the system searches the patient’s records (using a vector database of document embeddings) to locate the original source text that supports that fact. Essentially, the AI is asked: “Where did this piece of information come from?” Every fact must be matched to a snippet in the records (for example, the patient’s age is verified from the admission note, a lab value from the lab report, etc.).
4. Verification — A second LLM then compares each fact to the retrieved source text and scores how well they align. It checks that the fact is truly supported by the source and not a misunderstanding or fabrication. Mayo’s team even looked for a causal relationship — ensuring the context implies that fact, not just a coincidental mention.
5. Output with References — Only facts with solid support are kept. The final output is delivered with inline citations or links to the original records for every data point. This means physicians can click a link and see exactly where each piece of information came from, ensuring transparency and trust.