In June 2023, a federal judge in Manhattan sanctioned two attorneys and their firm $5,000 for a filing that cited six court decisions that did not exist. The lawyer who drafted it, Steven Schwartz of Levidow, Levidow & Oberman, had asked ChatGPT for supporting precedent in a personal-injury suit against the airline Avianca — and ChatGPT obliged with convincingly formatted, entirely fabricated cases, complete with invented quotes and docket numbers. When opposing counsel couldn’t locate “Varghese v. China Southern Airlines,” the unraveling began.
Mata v. Avianca became the most famous cautionary tale in legal AI — and taught the profession exactly the wrong lesson.
The Lesson Everyone Learned
Hallucination. Every bar journal, CLE program, and law-firm memo that followed focused on verification: AI invents citations, so check everything it produces. Judge P. Kevin Castel’s sanctions opinion made the point crisply — there is nothing “inherently improper” about using AI for assistance, but lawyers remain responsible for what they file.
That lesson is correct, and by now, widely absorbed. Courts began requiring AI-use certifications. Firms added verification policies. The output side of the problem got a governance framework.
The Lesson Almost Everyone Skipped
Ask a different question about the Avianca affair: what did the lawyers send in order to get those fake cases back?
Legal research prompts don’t happen in the abstract. To get useful output, a lawyer describes the client’s situation — facts, parties, injuries, jurisdictions, procedural posture. In other matters it’s worse: attorneys paste contract language for review, discovery documents for summarization, demand letters for response drafts, deposition excerpts for analysis. Every one of those prompts can carry client confidences protected under Model Rule 1.6, work product, or material covered by attorney-client privilege.
Unlike the fake-citation problem, this one produces no embarrassing court order. It’s silent. A privileged document pasted into a consumer AI account is a potential confidentiality breach — and arguably a privilege-waiver argument waiting for a motivated adversary — whether or not anything ever “goes wrong” visibly.
The profession’s institutions have caught up on paper. The ABA’s Formal Opinion 512 on generative AI (July 2024) put confidentiality at the center: lawyers must evaluate whether client information entered into an AI tool is adequately protected, and in many configurations that means informed client consent before inputting confidential information at all. State bars — California and Florida among the first — issued guidance in the same spirit: competence, confidentiality, supervision.
But guidance governs what lawyers should do. On a Thursday night before a filing deadline, what a tired associate actually does is paste the whole document.
Why Law Firms Are a Hard Case
Three features make legal work unusually exposed to prompt-level leakage:
- The work product is confidential by default. A marketing team’s drafts are mostly destined for the public. A law firm’s drafts are almost never. There is no “safe” category of client document to paste casually.
- Identifiers are load-bearing. Party names, case numbers, and specific facts are what make legal AI output useful — and exactly what makes the prompt a confidentiality event. De-identifying properly takes longer than the AI saves.
- Privilege is fragile. Confidentiality obligations are ethical; privilege is evidentiary. Disclosure to third parties is the classic way privilege dies, and “we routed it through a consumer chatbot” is not a sentence any litigator wants to defend.
How Blacksight Fits a Law Firm
Blacksight was built with a legal policy template precisely because this vertical’s patterns are distinctive. Out of the box it detects privilege markers (“PRIVILEGED,” “ATTORNEY-CLIENT,” “WORK PRODUCT”), case and docket numbers, and settlement amount references — alongside the universal detectors for PII, financial data, and credentials. Firms add custom rules for client names and matter numbers.
Enforcement matches legal reality: privileged-marked content blocks outright; case identifiers and settlement figures can redact in place so the associate still gets their summary — minus the parts that waive privilege. Everything scans locally on the lawyer’s device; prompt text never reaches Blacksight’s servers, so the confidentiality tool isn’t itself a disclosure. And every verdict lands in an audit trail — the documentation of “reasonable efforts” that Rule 1.6 analysis and client security questionnaires increasingly demand.
The associate keeps the productivity. The client keeps the privilege. Start free on 5 devices — smaller firms are running policy-enforced AI the same afternoon.
Frequently Asked Questions
Can lawyers use ChatGPT ethically?
Yes, within limits the bar has now mapped: verify all output (the Avianca lesson), and protect client confidentiality on the input side — ABA Formal Opinion 512 requires evaluating whether client information is safe to enter at all, often requiring informed consent. Practically, that means keeping identifying and privileged material out of prompts, ideally enforced by tooling rather than willpower.
Does pasting client documents into ChatGPT waive privilege?
It creates a genuine risk that an adversary will argue exactly that — voluntary disclosure to a third party is the standard path to waiver, and consumer AI tools retain data under terms the lawyer doesn’t control. No firm wants to litigate the question with its own conduct as the test case.
What happened in Mata v. Avianca?
Attorneys filed a brief citing six nonexistent cases generated by ChatGPT. In June 2023, Judge P. Kevin Castel sanctioned attorneys Steven Schwartz, Peter LoDuca, and their firm $5,000 jointly, finding they abandoned their responsibilities — while noting AI assistance itself is not inherently improper. The case became the defining warning about unverified AI output in court filings.