How AI Reads a Contract: From Raw Text to Risk Flags — A Lawyer's Guide

When you drop a forty-page agreement into an AI contract review tool and get back a list of flags in seconds, it is tempting to treat the result as if the machine read the contract the way you would. It did not. Between the upload and the verdict sits a pipeline of discrete stages — and understanding that pipeline is the difference between a lawyer who trusts the output and one who can defend it.

The details vary from tool to tool — CoCounsel, Lexis+ AI, Harvey, Luminance, and Lawyer Assistant all implement variations — but the anatomy is the same. Here is what happens, stage by stage, in plain English.

Stage one — making the document readable

Many contracts arrive as scanned PDFs: pages of pixels, not text. The first stage is optical character recognition (OCR), which converts the image into machine-readable text while trying to preserve the layout — headings, paragraphs, tables, signature blocks. If this stage is sloppy, everything downstream inherits the mistake. A clause split by a bad OCR boundary will be searched and classified incorrectly, no matter how good the model is.

Stage two — cutting the document into searchable pieces

A language model cannot comfortably hold a forty-page agreement in its working memory, and searching a whole contract at once is noisy. So the document is divided into passages — chunks cut at logical boundaries such as sections and paragraphs, rather than fixed character counts. Chunking quality matters more than it looks: the goal is to keep a legal unit intact. An indemnification obligation separated from its limitation-of-liability cap reads like a different clause than the one the parties actually signed.

Stage three — finding the passages that matter

Now the tool has to find the right passages for the question at hand. This is hybrid search, and it runs two retrievers at once:

• Keyword search — exact matching, in the tradition of BM25 ranking. This catches precise identifiers: defined terms, section numbers, "termination for convenience," a specific dollar cap. Legal drafting is full of this vocabulary, and exact matching never misses it.
• Semantic search — embeddings that map words and concepts into a space where meaning is proximity. This catches the paraphrase: "the vendor shall make good any loss" is retrieved even when the query says "indemnification."

The two result sets are fused and then reranked by a second model that scores each candidate passage against the question directly, so the top three to five passages that reach the language model are the ones most likely to matter. This is the same retrieval machinery behind a good legal research tool — the difference is that here, the corpus is your contract.

Stage four — classifying the clauses

With the right passages retrieved, the model labels each one by clause type, using taxonomies built from expert-annotated contract datasets — the CUAD benchmark being the best known. Indemnification. Limitation of liability. Assignment and change of control. Termination. Confidentiality. Governing law. This classification is the foundation of everything that follows: you cannot flag a deviation from your standard until you know what kind of clause you are looking at.

Stage five — comparing against your playbook

A classification on its own is just a table of contents. The value appears when it is compared against your standard. A playbook encodes the firm's position for each clause type: the baseline or preferred language, pre-approved fallbacks, hard walkaway lines, and who owns each escalation. The AI checks every classified clause against these rules and emits risk flags with severity ratings and explanations. A ten-page agreement becomes an audit against your own rules — no clause missed, no deviation unquantified.

Stage six — grounding every claim in the source

Finally, the model answers only from the retrieved passages — not from general knowledge — and every answer is tied to the exact source chunk with an inline citation. This is the "grounded" part of retrieval-augmented generation, and it is the feature that makes verification possible at all: you can open the flagged clause in the original document and judge it yourself. The pipeline is designed so that nothing reaches you without a paper trail.

Where it still gets it wrong

Grounding reduces hallucination; it does not eliminate it, and the research is honest about that. A widely cited 2024 study from Stanford's RegLab and HAI — Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools — tested leading commercial legal AI tools and found they returned incorrect or misgrounded answers in more than 17% of queries (one major tool above 34%), distinguishing between incorrect statements of law and misgrounded responses, where the rule is right but the cited source does not support it. Earlier work from the same Stanford group found general-purpose models answered legal research questions from memory with hallucination rates of 58–82%.

The courts have noticed. In Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023), Judge Castel imposed a $5,000 sanction on lawyers who filed a brief containing six fabricated cases generated by ChatGPT. In Park v. Kim, 91 F.4th 610 (2d Cir. 2024), the Second Circuit referred an attorney to its grievance panel for submitting an appellate brief with a hallucinated case. The pattern is not a curiosity; it is a professional-responsibility trend.

Which is exactly why the ethics framework matters. ABA Formal Opinion 512 (July 2024) maps generative AI onto the core rules: competence (Rule 1.1) requires understanding the tool's capabilities and limits; confidentiality (Rule 1.6) governs what you may enter into a tool; candor to the tribunal (Rule 3.3) requires verifying every citation before it is filed; and supervision (Rules 5.1 and 5.3) covers everyone in the firm who uses AI. The takeaway for review work: a flag from the machine is a hypothesis, and verification is the practice of law.

A review workflow that respects the pipeline

1. Define the playbook first. The audit only means something against your standard, so write the standard before you scan.
2. Let the machine scan. With the playbook as the rulebook, every clause is classified and compared; deviations come back flagged and rated.
3. Open every flag against the source. The citation is the feature. Confirm the flag is accurate, judge its materiality in context, and decide: resolve, dismiss, or escalate.
4. Negotiate with the record. A documented, portfolio-wide finding — "all six supply agreements carry unlimited liability language" — is a stronger position than an impression.

Notice the division of labor, and why it is defensible: the machine guarantees completeness and consistency; the lawyer supplies context and judgment. Each stage of the pipeline exists to make the other possible.

How Lawyer Assistant implements this

Lawyer Assistant is built on the same anatomy: hybrid search (BM25 + dense vectors, fused and reranked) retrieves the passages, and a compliance playbook scan classifies clauses and checks them against your own rules, with severity ratings and explanations. Every flag links to the source text so verification is a click away. And because everything runs locally on your machine, the confidentiality half of ABA 512 — what you enter into the tool — is handled by architecture rather than by promise. More on privilege and local processing here.

The bottom line

AI does not read contracts; it makes them searchable, classifies their parts, and compares them against a standard you set — then hands you a cited list of everything you need to look at. The machine has no opinion about what the parties meant, what the jurisdiction will enforce, or how hard to push back in the negotiation. Those remain the lawyer's work, and they are precisely the parts that cannot be automated.

"The tool guarantees nothing gets missed. The lawyer decides what matters."

Sources & further reading

• Magesh, Surani, Dahl, Suzgun, Manning & Ho, Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (Stanford RegLab / HAI, 2024).
• Dahl, Magesh, Suzgun & Ho, Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models (Stanford, 2024) — 58–82% hallucination rates on legal queries.
• Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023).
• Park v. Kim, 91 F.4th 610 (2d Cir. 2024).
• ABA Formal Opinion 512, "Generative Artificial Intelligence Tools" (July 29, 2024) — full text (PDF).
• Hendrycks et al., CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review (2021).

This article is general information about technology and professional practice. It is not legal advice for any specific matter, and rules vary by jurisdiction — verify against the authority applicable to your matter.