Can You Trust an AI for Legal Research? Hallucinations, Citations & Verification

In 2023, a federal judge sanctioned two lawyers and their law firm after they submitted a brief citing six fabricated judicial opinions — including a case called Varghese v. China Southern Airlines that did not exist, and a "quote" from a Ninth Circuit opinion that no one could find. The lawyers had asked ChatGPT to find case law. It obliged. Every citation was invented.

The case, Mata v. Avianca, became the cautionary tale of legal AI. But it did not make lawyers stop using AI — it made them ask a sharper question: how do I know whether the answer is true?

Why an AI "hallucinates" case law

A large language model does not look anything up. It generates text by predicting, word by word, what is most plausible given everything it has read. When you ask it for a case, it produces what looks like a case: a name, a citation format, a holding. The model is fluent about a thing it has never retrieved.

This is not a bug that can be tuned away with a better prompt. It is the difference between generation and retrieval — between a writer with a vivid imagination and a librarian who pulls the actual book off the shelf. That distinction is the whole story of trustworthy legal AI.

Grounded retrieval: the fix that actually works

The industry's answer is Retrieval-Augmented Generation (RAG). Instead of letting the model answer from memory, a RAG system:

1. Searches first — it queries an index of real documents (case files, statutes, your firm's own briefs) for passages relevant to the question.
2. Retrieves the source text — the actual sentences, with file and page information, not a paraphrase.
3. Answers only from what was retrieved — the model is instructed that the retrieved passages are the only permissible evidence, and every claim must cite one.

The result is a fundamentally different kind of answer: every assertion carries a pointer to a passage you can open and read. This is the single most important feature to look for in any legal AI tool. A chatbot that answers from memory is a drafting assistant. A grounded system that answers from your documents is a research assistant.

The quality of that answer, though, depends entirely on the quality of the search step. This is where hybrid retrieval matters — combining exact keyword matching (the way Westlaw and Lexis always worked, catching precise legal shorthand and citations) with semantic search (finding meaning even when the words differ) and then reranking the results so the strongest passages surface first. A tool that only does one kind of search will miss answers a lawyer would find in five minutes.

What a trustable answer actually looks like

Here is the practical test, and it works no matter which tool you use:

• Every claim has a citation. No citation, no reliance. This is not negotiable.
• The citation opens a real source. You can click it and see the exact passage, on the page, in the document.
• You can read the retrieval yourself. The best tools show you which documents and passages the AI actually used — so you can judge whether the source is on point before you judge the answer.
• The tool can say "I don't know." When nothing relevant is found, a trustworthy system tells you, instead of improvising from general knowledge.

"Treat an AI answer the way you'd treat a memo from a first-year associate: read the cases it cites before you sign anything."

The verification workflow: draft with AI, check like a lawyer

The safest pattern is a two-tier workflow, and it maps exactly onto how good lawyers already work:

1. Use the AI to surface — to find the passages, statutes, and arguments that are plausibly relevant to your question. This is where AI saves hours.
2. Use your judgment to conclude — read the cited passages, confirm they say what the answer claims, check the date and jurisdiction, and decide what actually applies to your matter.
3. Document your verification — if you relied on an AI research tool, you should be able to say exactly which sources you checked and how. Courts are increasingly asking.

Notice what this workflow doesn't include: asking a general chatbot to invent case law, citing an answer without reading its sources, or treating an AI summary as a substitute for primary authority. Those are the habits that produce Mata v. Avianca stories.

How Lawyer Assistant approaches this

Lawyer Assistant is built around the grounded-retrieval model described above, on purpose. It searches your documents — PDFs, scans, contracts, case files — with hybrid retrieval (BM25 keyword matching plus dense semantic vectors, then reranking), and it answers only from what it finds. Every answer streams in with inline citations linking to the exact source passages, and a Sources panel lists every cited document with file, page, and section. When the retrieval finds nothing relevant, it says so instead of guessing.

Two details make this verifiable rather than just pretty: the pipeline editor lets you see exactly which stages ran on your question (search, rerank, answer), and the chatbot flow is documented end to end — from message to intent routing to retrieval to final answer. You are not asked to trust a black box; you are shown the chain of evidence and invited to check it.

The bottom line

Can you trust an AI for legal research? Yes — with the same caveats you apply to any research source: verify the authority, check the citation, and keep your own judgment in the driver's seat. The tools that deserve your trust are the ones that refuse to answer from memory, that cite what they retrieve, and that let you look behind the answer.

If a tool cannot show you its sources, it is not doing research. It is doing creative writing with a legal vocabulary.