Robert Ambrogi Blog
Which AI models are actually best at legal work? A new platform launched in beta by the alternative legal services provider Percipient aims to answer that question by letting legal professionals put the models to the test themselves, in blind, head-to-head comparisons, and all at no cost.
The platform, Certera.AI, lets a user submit a legal prompt and receive answers from two AI models displayed side by side. In its anonymous mode, the platform hides the models’ identities until after the user votes for the response they prefer (or votes that both were good or both were bad) so that the judgment turns on the quality of the answer rather than the brand behind it.
Users can also select specific models to pit against each other, or query a single model, and they can attach documents such as contracts for review or redlining.
All those votes get aggregated by Certera to create a public leaderboard that ranks the models on legal work, which Percipient founder and CEO Chad Main told me he expects to go live this week.
“That’s ultimately our goal — to be the place to go where you can figure out which models, just off the shelf, are best for whatever area of law you’re trying to work in,” Main said.
Legal Only, Vetted Voters
Blind arena-style model comparison is not a new idea. Sites such as Arena (formerly LMArena) popularized the format for general-purpose prompting. But Main said a common critique of those general sites is that anyone can vote, making the resulting rankings hard to trust.
“It’s sometimes hard to really trust the results and the leaderboard, because we don’t know who’s prompting,” he said.
Certera’s answer is to restrict access to verified legal professionals. The signup process asks attorneys for their bar numbers, and law students and other legal professionals can also register. For now, Main said, Percipient is vetting registrants manually, although he is working on a more seamless verification method for when user numbers grow.
The leaderboad is not yet showing rankings, but the rankings should start showing up soon.
Certera passes prompts through to the models untouched, with no system prompt or other modification layered on top.
“I purposely told our development team, we don’t adulterate that prompt at all,” he said. “Whatever you’re getting is straight from the AI’s API, because we want honest opinions about what lawyers think of these models.”
The platform currently includes 55 models, spanning the major U.S. labs as well as open source and Chinese models. New releases get added quickly, Main said. The latest GPT and Grok models went in within days of their release. One notable absence is Meta, whose models Main said have not been updated in some time.
For now, the lineup consists only of general-purpose foundation models, not legal-specific products, although Main said he plans to invite legal AI vendors to connect their tools via API.
“I hope they would,” he said, “because they can put their money where their mouth is.”
An Elo Leaderboard
The leaderboard ranks models using the Elo rating method, the head-to-head system originally developed to rank chess players and since adopted by Arena and others for model comparison. Certera updates its rankings every evening.
To seed the system with enough votes for the rankings to stabilize, Percipient has supplemented user voting by running public data sets – case law questions with verifiable yes-or-no answers, citation checks, and bar exam questions – through the platform.
Main gave me a preview of the leaderboard during our call. The results were still preliminary, and he said Percipient is working to narrow the margin of error, which will require more votes from real users.
In my test, one result was clearly better, so I chose column B.
But some early patterns caught his attention. OpenAI’s newly released GPT-5.6 debuted at the top of the rankings, and some of the Chinese open source models, including Qwen and GLM models, have performed surprisingly well.
The Elo ranking is strictly head to head and does not distinguish among types of legal work. But Main said Percipient is working with a statistics professor at Indiana University on a method to rank models by context – so a user could see, for example, that one model leads on contract law questions while another leads on litigation or discovery. He hopes to roll that out within a few weeks.
Because Certera tracks the area of law each prompt relates to, the company also plans research into specific tasks, starting with which models perform best and worst on case law.
An open question that shadows all preference-based rankings, Main said, is what voters are actually rewarding. On general-purpose arenas, he noted, research suggests that a better-formatted answer often beats a more accurate one.
“If it’s prettier, it gets votes over what may be the more accurate answer,” he said. “So it’s going to be interesting to see how legal professionals vote. Are they voting on thoroughness? Is one more technically right but didn’t follow the instructions to a T? Which is preferred?”
Free to Use
Certera is free for legal professionals, with Percipient covering the API costs, and Main said there are no plans to charge users.
There is, however, a business rationale. About six months ago, Percipient, which continues its core ALSP work in managed review, contract review and related services, expanded into providing evaluation data and expert feedback to AI companies to help them train and improve their models.
Last month, it released a benchmark study, “How Frontier AI Models Perform on Real Legal Work,” in which experienced attorneys graded frontier models on practical legal deliverables using detailed rubrics.
With the launch of Certera, Main said he hopes eventually to offer the platform as a testing service to AI companies that want to evaluate a model on legal tasks before release, and to legal AI vendors that want to benchmark their products against the frontier models.
Legal professionals using the platform will appreciate that it includes an anonymization feature that detects potentially sensitive information in a prompt — names, Social Security numbers, tax IDs — and offers to strip it out with one click before the prompt is sent. Even so, Main cautioned that users should not put genuinely sensitive client information into the platform.
Giving It A Try
Ahead of my call with Main, I gave Certera a quick spin the night before, and my first result was a mixed bag. One model stalled and never delivered an answer. (Main said that is a hazard of relying on the models’ own APIs.) The other confidently answered my Massachusetts legal question by citing a statute as controlling that had nothing to do with the issue.
However, in that first test, even though I voted that both answers were bad, it never revealed the models’ names, perhaps because the one had seemed to freeze.
Once I made my choice, the names of the models were revealed.
This morning, I went back to the site, completed the registration (which asked for my bar number to confirm I am a lawyer), and tried it again.
This time, I again asked a Massachusetts legal question — one for which I am well-versed in the answer, so I could quickly evaluate the results.
This time, both LLMs gave me the right answer. But one was notably better, with a more detailed analysis and discussion of applicable cases. It even flagged that there is pending legislation that would set a statutory standard.
When I indicated my preference for that answer, Certera then revealed the names of the two products. The winner in my test was Claude Fable 5. The so-so answer came from Grok-4.20 Reasoning Beta.
So now I have added two votes to the pile. I am hoping there is soon enough of a critical mass for the leaderboard to start displaying results.
To that end, I recommend you go ahead and try it yourself. It costs nothing. And it is kind of fun.