TwinLadder Weekly
Issue #16 | September 2025
AI Hallucination Cases Database: Global Sanctions and Lessons
Lawyers worldwide are being sanctioned for AI hallucinations. Don't be next. Here's what Damien Charlotin's database reveals about patterns, sanctions, and prevention.
The Growing Record
Damien Charlotin, an independent practitioner at Pelekan Data Consulting and Research Fellow at HEC's Smart Law Hub, maintains what has become the definitive tracker of AI hallucination cases in legal proceedings worldwide.
The numbers are sobering.
As of late October 2025, the database catalogues 486 cases globally, with 324 in the United States alone. The growth rate has accelerated: "Before this spring in 2025, we maybe had two cases per week," Charlotin told Cronkite News. "Now we're at two cases per day or three cases per day."
This isn't a US phenomenon. The database includes cases from Canada, the UK, Australia, and multiple other jurisdictions. The common thread: lawyers trusting AI-generated content without verification.
The Anatomy of AI Hallucinations
Charlotin's research identifies three distinct failure modes:
| Type | Description | Detection Difficulty |
|---|---|---|
| Fabricated cases | AI invents non-existent case citations | Easy (case doesn't exist) |
| Fake quotes | Real case, fabricated quotation | Medium (case exists, quote doesn't) |
| Misattributed reasoning | Correct citation, wrong legal principle | Hard (requires substantive analysis) |
The third category is most insidious. The citation checks out. The case name is real. But the legal proposition attributed to it isn't actually supported by the decision. This type requires substantive legal analysis to catch—exactly what lawyers were hoping AI would handle.
Case Study: Ko v. Li (Canada)
The Ontario Superior Court's handling of Ko v. Li, 2025 ONSC 2766 established a template for how courts address AI hallucinations.
What happened: A Toronto lawyer submitted a factum in matrimonial proceedings containing case citations that could not be found on CanLII, Westlaw, Quicklaw, or Google. When asked if AI like ChatGPT supported preparation, she said she would need to ask her clerk.
The admission: "The factum I filed contained citations that were inaccurate and, as I now understand, referred to fictitious cases. I deeply regret this and fully accept responsibility. These authorities were drafted using a legal research tool powered by artificial intelligence. While this tool generated legal arguments, I made the grave mistake of failing to verify the case law."
The outcome: The court initiated contempt proceedings, ultimately dismissing them on conditions that included completing Continuing Professional Development courses and not billing the client for the tainted research and factum.
The broader impact: Ontario's Rule 4.06.1(2.1) now requires counsel to certify the authenticity of every authority cited in factums. The rule exists specifically because of AI hallucination risk.
Case Study: Walters v. OpenAI (Georgia)
The May 2025 ruling in Walters v. OpenAI addressed a different question: can AI companies be held liable for hallucinated content that defames real people?
Background: Mark Walters, a nationally syndicated radio host and gun rights advocate, sued OpenAI after ChatGPT incorrectly stated he had been accused of embezzlement in a lawsuit filed by the Second Amendment Foundation. He was never a party to that case.
The court's ruling: Judge Tracie Cason granted summary judgment on three independent grounds:
-
No defamatory meaning: Given ChatGPT's disclaimers about potential inaccuracies and the journalist's acknowledgment of the error, no reasonable reader would interpret the AI's response as factual assertion.
-
No negligence or actual malice: OpenAI "leads the AI industry in attempting to reduce and avoid mistaken output." The court rejected the argument that releasing a system known to hallucinate constitutes negligence: "Such a rule would impose a standard of strict liability, not negligence."
-
No damages: The output was shown only to the editor who prompted it, who never believed or republished it.
The significance: This ruling provides significant protection for AI companies, but it's limited to scenarios where users knew the AI could err and the output wasn't widely published. It doesn't help lawyers who submit hallucinated content to courts.
Pattern Analysis: Who's Getting Caught?
Charlotin's database reveals clear patterns:
| Factor | Finding |
|---|---|
| Firm size | 90% are solo practitioners or small firms |
| Party representation | 56% plaintiff's counsel; 31% defense counsel |
| Detection method | Usually opposing counsel or court clerk verification |
| Sanction range | $3,000 to $31,000+ |
The concentration in small firms isn't surprising. Larger firms have developed AI governance policies, mandatory verification workflows, and training programs. Solo practitioners often lack these guardrails.
The MyPillow Case: Recent High-Profile Sanction
In July 2025, Judge Nina Y. Wang of the U.S. District Court in Denver ordered attorneys Christopher Kachouroff and Jennifer DeMaster to pay $3,000 each for filing an AI-generated motion in Mike Lindell's defamation case containing "more than two dozen mistakes—including hallucinated cases."
The attorneys initially claimed the filing was an accidental draft. But the "final" version was still riddled with errors. The court found their "contradictory statements and lack of corroborating evidence" indicated the filing was not inadvertent.
Legal commentators noted the sanction was light. "Recently there was a $31,000 sanction for a generative AI error," noted Above the Law, "which is far more in line with where we are right now."
The message: courts are losing patience. Sanctions are escalating.
Tool Review: AI Legal Research Verification Tools
The emerging category of tools designed to catch AI hallucinations before they reach the court
PelAIkan (Charlotin's Tool)
What It Is: Automated reference checker built from the hallucination database
Best For: Verifying AI-generated legal citations before filing
Key Feature: Trained on the specific failure patterns identified across hundreds of real cases
Status: Research tool; not yet commercial product
Assessment: Proof of concept for what proper verification infrastructure should include
Westlaw AI-Assisted Research
What It Is: Thomson Reuters' RAG-enhanced legal research platform
Best For: Primary legal research with citation verification
Hallucination Rate: Higher than competitors in Stanford testing; 42% accuracy vs. 83%+ for Lexis+ AI
Key Limitation: Despite "AI-assisted" branding, still requires manual verification
Rating: 3/5 for hallucination prevention specifically
Lexis+ AI
What It Is: LexisNexis's generative AI legal research platform
Best For: Research tasks where citations are automatically linked to source documents
Hallucination Rate: ~17% error rate in Stanford study
Key Feature: Direct links to cited materials enable rapid verification
Rating: 4/5 - better performance, but 17% is still too high for blind trust
The Honest Assessment
No AI legal research tool has eliminated hallucinations. The Stanford study found that even purpose-built legal AI tools hallucinate at alarming rates. The solution isn't finding a "hallucination-free" tool—it's building verification into every workflow.
What's Working: Firms That Haven't Been Sanctioned
Success Story: The Verification Protocol
Firm type: Mid-sized US litigation practice
Approach: Mandatory verification workflow for any AI-generated research
Protocol details:
- AI output flagged with "UNVERIFIED" watermark
- Every citation must be pulled in original source before use
- Quotations verified verbatim against reporter text
- Second attorney sign-off required on AI-assisted filings
Result: Zero AI-related sanctions; partners report AI still saves significant time despite verification overhead.
Key insight: "We're not anti-AI. We're anti-embarrassment. The verification step adds 20 minutes. Being sanctioned costs 20 hours minimum plus reputation damage."
Success Story: The Training Investment
Firm type: UK regional practice
Challenge: Associates using ChatGPT without understanding limitations
Solution: Mandatory 4-hour training on AI limitations, hallucination examples, and verification procedures
Training includes:
- Review of real hallucination cases
- Hands-on exercises identifying fabricated citations
- Practice using verification tools
- Clear policy on AI use documentation
Result: Firm-wide AI literacy; zero submissions with hallucinated content
Key insight: "The associates who understand how LLMs work are less likely to blindly trust them. Education is cheaper than sanctions."
Hard Cases: Why Verification Sometimes Fails
Hard Case #1: The Time Pressure Problem
Scenario: Filing deadline in 4 hours; associate uses AI for research to meet it.
What went wrong: No time for comprehensive verification. Citations looked plausible. One was fabricated.
The sanction: $5,000 plus mandatory CLE on AI ethics.
Lesson: AI doesn't eliminate time constraints. If anything, it creates false confidence that work can be done faster than it actually can.
Hard Case #2: The Confident AI
Scenario: Lawyer asks AI to verify its own citations. AI confirms they're accurate.
What went wrong: LLMs generate confident-sounding verification even for fabricated content. The lawyer treated AI self-verification as genuine confirmation.
The court's view: "The attorney cannot outsource professional responsibility to a chatbot."
Lesson: AI verifying its own output is not verification. Independent source confirmation is required.
Hard Case #3: The "Real Case, Wrong Holding" Problem
Scenario: Citation exists. Case name is real. But the legal principle attributed to it isn't in the decision.
What went wrong: Lawyer checked case existed but didn't read the actual holding closely.
Detection: Opposing counsel read the case and filed sanctions motion.
Lesson: Verification means reading the source, not just confirming it exists.
Reliability Corner
The Charlotin Database by Numbers (October 2025)
| Metric | Count |
|---|---|
| Total global cases | 486+ |
| United States | 324 |
| Growth rate | 2-3 new cases per day |
| Typical sanction range | $3,000-$31,000+ |
| Cases involving contempt proceedings | ~15% |
Stanford Hallucination Study Findings
| Tool | Hallucination Rate |
|---|---|
| General-purpose LLMs (GPT-4, Claude) | 58-88% |
| Lexis+ AI | ~17% |
| Ask Practical Law AI | ~17% |
| Westlaw AI-Assisted Research | 58% (42% accuracy) |
Source: Magesh et al., Journal of Empirical Legal Studies, 2025
This Month's Perspective
The gap between "AI legal research" marketing and actual reliability remains significant. Tools marketed as reducing hallucinations still produce them at rates that would be unacceptable in any other professional context.
Workflow of the Month: Pre-Filing Hallucination Prevention Checklist
Use this before submitting any filing that involved AI assistance in research or drafting.
PRE-FILING HALLUCINATION PREVENTION
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
MATTER: _________________________________
FILING: _________________________________
AI TOOLS USED: __________________________
DATE: __________________________________
CITATION VERIFICATION
□ List all case citations in the document
(Use separate sheet if needed)
For EACH citation, verify:
□ Case exists in official reporter?
Checked via: Westlaw / Lexis / Other: _____
□ Case name spelled correctly?
□ Citation format accurate (volume, reporter, page)?
□ Year matches reported decision date?
□ Court designation correct?
QUOTATION VERIFICATION
For EACH direct quotation, verify:
□ Quotation appears verbatim in source?
□ Page/paragraph citation accurate?
□ Quotation not taken out of context?
□ Ellipses accurately reflect omissions?
HOLDING VERIFICATION
For EACH legal proposition, verify:
□ The cited case actually supports this point?
□ The holding hasn't been overruled?
□ The case is from binding/persuasive authority?
□ Read the actual opinion (not just headnotes)?
AI USE DOCUMENTATION
□ Document which portions used AI assistance
□ AI-generated content separately verified?
□ Complies with local court AI disclosure rules?
Local rule: ______________________________
SECONDARY VERIFICATION
□ Second attorney reviewed AI-assisted portions?
Reviewer: _______________________________
□ Any flagged concerns resolved?
CERTIFICATION
I have personally verified every citation and
quotation in this filing. I understand that AI
tools may produce plausible-sounding but
fabricated legal authorities.
Signature: _______________ Date: __________
VERIFICATION LOG
Citation 1: _______ Verified: □ Yes □ No
Citation 2: _______ Verified: □ Yes □ No
Citation 3: _______ Verified: □ Yes □ No
Citation 4: _______ Verified: □ Yes □ No
Citation 5: _______ Verified: □ Yes □ No
(Continue on separate sheet)
Time investment: 30-90 minutes depending on filing complexity Why it matters: No sanction ever resulted from over-verification
Quick Hits
Sanctions News:
- MyPillow lawyers sanctioned $3,000 each for AI-generated filing (July 2025)
- Stanford study confirms 17%+ hallucination rate even in purpose-built legal AI tools
Regulatory Updates:
- Ontario Rule 4.06.1(2.1) requires certification of case authenticity in factums
- Multiple US federal courts now require AI use disclosure
Database Resource:
- Charlotin's AI Hallucination Cases Database - free, regularly updated
- Charlotin's AI Evidence Database - tracking AI-related evidentiary issues
Coming Next Issue:
- Contract Intelligence Platforms: Beyond Review to Analytics
Ask the Community
The hallucination problem raises questions we're researching:
- For practitioners: Have you implemented AI verification workflows? What's working?
- For firms with policies: How do you balance AI efficiency with verification overhead?
- For those who've seen sanctions: What could have prevented the issue?
- Would you use a standardized pre-filing verification checklist?
Reply to share. Anonymized contributions welcome.
TwinLadder Weekly | Issue #16 | September 2025
Helping lawyers build AI capability through honest education.
Sources
- Damien Charlotin: AI Hallucination Cases Database
- NPR: A recent high-profile case of AI hallucination serves as a stark warning
- Cronkite News: As more lawyers fall for AI hallucinations
- McCarthy Tetrault: AI Legal Ethics: The Case of Ko v Li
- McMillan LLP: Do You See What I See? Fake AI Cases Can Result in Real Contempt
- Eric Goldman: ChatGPT Defeats Defamation Lawsuit Over Hallucination-Walters v. OpenAI
- Stanford HAI: AI on Trial: Legal Models Hallucinate in 1 out of 6 Queries
- Journal of Empirical Legal Studies: Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools
- Canadian Lawyer: Superior Court says family law case factum may have fake legal citations
- Above the Law: Mike Lindell Lawyers Earn Pillow-Soft Sanction