Robert Ambrogi Blog
With more than 650 documented cases of AI hallucinations appearing in court filings, and courts imposing sanctions ranging from tens of thousands of dollars to removal from client representations, law firm partners face an uncomfortable quandary: How can they confidently sign pleadings when they cannot be certain whether someone on their team used AI tools that might have introduced fabricated citations?
Aiming to give partners the documented assurance they need before they sign their names to a brief, Clearbrief this week launched Cite Check Report, a new feature that provides an audit trail showing that every citation in a document has been systematically verified – including both legal and factual citations.
“Partners are being sanctioned and suffering reputational damage for citation errors they didn’t personally make,” said Clearbrief CEO Jacqueline Schafer. “We built the Cite Check Report to give them what courts are demanding: documented proof that they satisfied their ethical obligations before signing that pleading.”
The Hallucination Problem
The launch reflects both the increasing anxiety of legal professionals around hallucinations and the increasingly severe consequences. In one recent hallucinations case, a court ordered the attorney to pay $85,000 in sanctions and attorneys’ fees. In other cases, judges have required sanctioned attorneys to notify all their clients about their AI citation failures.
The problem, Schafer points out, is not limited to rogue attorneys knowingly using unreliable AI tools. In many of the sanctions cases, partners had no idea AI was even used. It was someone else on the team who used the AI, sometimes surreptitiously, but sometimes within the firm’s own AI policy.
In some cases, attorneys have been sanctioned even though the AI tools they used were from what they considered to be trusted sources – and therefore assumed there would be no risk.
Another challenge is that AI-generated errors can be extremely subtle. “A lot of the errors that AI makes with citations are not easy to spot with the human eye,” Schafer said.
How the Cite Check Report Works
Clearbrief’s new report addresses these concerns by analyzing both legal authority citations and factual citations. Clearbrief says its product is the only citation analysis tool in the industry to cover both types of citations.
Working directly in Microsoft Word, the report automatically identifies all citations and flags potential issues, including:
- Missing cases or sources.
- Formatting errors.
- Low semantic scores, which can indicate the source does not support the writer’s assertion.
The system uses “classic” AI rather than generative AI, meaning there is no risk of the checking tool itself introducing hallucinations. The report includes hyperlinks to each citation, allowing the signing attorney to review any flagged issues in context.
The tool is able to verify citations regardless of format. While some competing tools cannot handle citations that are not in standard reporter formats – such as proprietary Westlaw (“WL”) and LexisNexis citations – Schafer says Clearbrief has solved this by mapping Westlaw citations to its caselaw databases from LexisNexis, Fastcase and vLex. And because Clearbrief integrates with LexisNexis, those citations are not an issue.
If Clearbrief cannot automatically display the case because it is proprietary to Westlaw or some other source, it will still detect that the citation can be verified. The user can then manually upload PDFs of cases to complete the verification.
Beyond citation format, Clearbrief checks whether cases actually support the propositions for which they are cited. The platform uses a patented semantic analysis score that compares each sentence to its cited source and flags low scores where the source does not appear to support the assertion. The tool also verifies the accuracy of quotations.
Providing Verifiable Proof
When a user finishes reviewing a document, generating the Cite Check Report is straightforward. The report allows the reviewer to add notes explaining their review process, such as why they may have ignored certain red flags.
It also tracks how associates, paralegals or attorneys addressed each identified issue, with any notes they left during their review.
The resulting PDF can be saved to the case file, creating a permanent record that demonstrates due diligence before filing.
This documentation matters, Schafer said, because the reality is that most partners rely on associates to check citations. In the past, trust was enough, she said, but in the AI era, partners need verifiable proof.
“It’s fast, it’s easy for the associate and the paralegal to do it for them,” Schafer said. “The partner doesn’t have to change anything, really. They just have that peace of mind now.”
The feature comes included with standard Clearbrief subscriptions, which already provide access to case databases through the company’s partnership with Fastcase/vLex. Users who also have LexisNexis subscriptions can access cases through Lexis as well.