In Mata v. Avianca, two experienced New York litigators submitted a brief citing over half a dozen cases that did not exist. The cases had been generated by ChatGPT, and the lawyers had not verified them against any legal database. When the fabricated citations were challenged, the lawyers doubled down, asking ChatGPT to confirm the cases existed rather than checking Westlaw or Lexis — and the AI confirmed them. The court imposed sanctions on the attorneys and their firm, citing a failure to meet basic professional obligations of competence and candor. Gurpreet Bal's reading of Mata is precise: the problem was not that AI was used for legal research. The problem was that AI output was submitted to a court as fact without any independent verification. The hallucination problem in general-purpose LLMs is well-documented and not surprising to anyone who has worked with these systems. What was surprising — and what the sanctions reflect — is that experienced lawyers apparently did not understand that AI systems can produce confident, plausible-sounding fabrications with no grounding in reality.
Following Mata, a reasonable lawyer might assume that purpose-built legal AI tools — Harvey, CoCounsel, Lexis+ AI, Westlaw AI — have solved the hallucination problem. Gurpreet S. Bal cautions against that assumption. These tools are significantly better than general-purpose LLMs at grounding legal citations in real cases, because they are trained on and retrieve from actual legal databases. But they are not infallible. Retrieval-augmented generation, the architecture underlying most legal AI tools, can still produce errors — misattributing a holding to the wrong case, citing a dissent as majority opinion, or confusing cases with similar names. Every legal research output from an AI tool, purpose-built or otherwise, requires citation verification before it appears in any filing, brief, or demand letter. This is not an indictment of the tools — they can dramatically accelerate research and surface relevant authorities that a manual search might miss. But the attorney submitting the filing remains responsible for every citation in it, and that responsibility cannot be delegated to software.
A distinct and growing challenge in litigation is the use — or alleged use — of AI-generated evidence. Deepfake audio and video are now sufficiently realistic that authentication of digital recordings has become genuinely contested in some cases. Courts are beginning to grapple with how to apply Federal Rule of Evidence 901 to AI-generated or AI-altered media, and the case law is still nascent. Gurpreet Bal sees this as a two-sided problem for corporate litigants: on offense, the risk that opposing counsel or an adversarial party introduces fabricated evidence that is difficult to detect; on defense, the risk that legitimately authentic evidence is challenged as AI-generated and the burden of proving authenticity is not straightforward. Several courts and commentators have proposed requiring disclosure when digital evidence has been processed, enhanced, or analyzed by AI tools, but no uniform standard exists yet. Companies involved in commercial litigation should be working with litigation counsel now to establish digital provenance practices for key evidence — particularly recordings, screenshots, and email chains — before disputes arise.
Gurpreet S. Bal draws a clear line in how AI should and should not be used in litigation. On the safe side of the line: large-scale document review and first-pass relevance coding, contract comparison across large data sets, deposition preparation assistance that identifies inconsistencies and suggests lines of questioning, timeline construction from thousands of documents, and summarization of deposition transcripts for attorney review. These are legitimate and powerful uses that reduce cost and improve quality. On the dangerous side: autonomous brief drafting submitted without full cite-by-cite verification, AI-generated expert analysis presented as the expert's independent work product, and agentic tools that file documents, serve process, or send communications without contemporaneous attorney sign-off. The most serious emerging risk is autonomous litigation agents — AI systems designed to manage litigation tasks end-to-end. Beyond UPL concerns, such systems introduce the possibility of AI-generated harassment, evidence fabrication, or procedurally improper filings at a scale and speed that human oversight cannot match in real time. The professional responsibility framework has not caught up to this risk, which means the liability for what an autonomous litigation agent does will fall — perhaps entirely — on the attorney or company that deployed it.
Gurpreet S. Bal is a Partner at Foley and Lardner LLP in Silicon Valley, where he advises technology companies, founders, and investors on corporate transactions and the evolving intersection of law and artificial intelligence. He has represented clients in hundreds of transactions with aggregate deal value exceeding $60 billion across AI, semiconductors, fintech, and emerging technology.