Robert Ambrogi Blog
Thomson Reuters says the large language model it has been quietly building in-house now performs competitively with the best general-purpose AI models in the world, and it has released its first benchmarking results to back up the claim.
In a July 31 post on the company’s innovation blog, Chief Technology Officer Joel Hron and Head of AI Research Jonathan Schwarz shared early benchmark data for Thomson, the eponymously named proprietary model the company has been developing since its 2024 acquisition of the AI research startup Safe Sign Technologies.
Across a range of legal and general-purpose benchmarks, they wrote, Thomson performed competitively with Anthropic’s Claude Opus 4.8 and ahead of OpenAI’s GPT-5.5, Anthropic’s Claude Sonnet 5, and Google’s Gemini 3.1 Pro.
“The most capable AI models no longer come only from frontier AI labs,” Hron and Schwarz wrote. “One now comes from Thomson Reuters.”
I’ll go into what the numbers actually show, how the tests were run, and what remains unverified, but first, some background on Thomson.
What Thomson Is
“Launching later this summer, Thomson is the newest layer of the Thomson Reuters AI strategy, and a demonstration of what becomes possible when authoritative content, expert judgment, professional tools, and model development come together,” Hron and Schwarz say in their post.
According to the post, Thomson starts from an open-source foundation model and is then trained, using what they describe as state-of-the-art mid-training and post-training techniques, on decades of proprietary content from Westlaw, Practical Law, Checkpoint and Reuters.
Hundreds of subject-matter experts evaluated outputs, identified failure modes, and validated the model’s reasoning, they say, adding that customer data is never used in training.
Notably, they say that less than 10% of Thomson Reuters’ proprietary content has been used in training so far – suggesting there is plenty of room for Thomson’s capabilities to further improve.
The model’s first deployment will come in August, when Thomson becomes the default model powering Tabular Analysis in CoCounsel Legal.
Hron and Schwarz say they chose Tabular Analysis because that type of high-volume, structured document review offers a clear, measurable accuracy standard where a purpose-built model’s advantage should be immediately visible. Over the next year, they write, Thomson Reuters will integrate the model across its legal and tax product portfolio.
The Numbers
The benchmark that drove the headline pits Thomson (technically called Thomson-1-Large) against Gemini 3.1 Pro, Claude Opus 4.8, and GPT-5.5 across three legal benchmarks and four general-domain composites. Thomson took the top score in three of the seven rows:
Legal domain:
- Stanford LegalBench: Thomson 0.823 – behind Gemini 3.1 Pro (0.843) and GPT-5.5 (0.832), ahead of Opus 4.8 (0.818).
- PrBench Legal Hard: Thomson 0.352 – best in class, ahead of GPT-5.5 (0.333), Opus 4.8 (0.315), and Gemini (0.293).
- Harvey Legal Agent Benchmark: Thomson 0.857 – a close second to Opus 4.8 (0.869), and well ahead of Gemini (0.555).
General domain:
- Instruction following (IFEval + FollowBench): Thomson 0.914 – best in class.
- Reasoning (GPQA Diamond, HLE, MMLU-Pro): Thomson 0.684 – behind Gemini (0.748) and Opus (0.737).
- Coding (SWE-Bench Pro, Terminal-Bench 2.1): Thomson 0.399 – last place, well behind Opus (0.598).
- Long context (Infinity Bench plus internal TR benchmarks): Thomson 0.753 – best in class, by a hair over Gemini (0.750).
In their blog post, Hron and Schwarz write, “Thomson matches the best. And beats the rest.” Taking the benchmark scores at face value, that is a fair characterization, particularly given that Thomson is a fraction of the size and operating cost of the frontier models it was up against. For a domain-specific model built by a legal information company, this is a notable achievement.
But the results are more mixed than the headline suggests. Thomson did not win a majority of the rows. Gemini and GPT-5.5 both beat Thomson in the best-known legal benchmark, LegalBench. Anthropic’s Opus edged Thomson on the Harvey agentic benchmark and beat everyone on coding.
In addition, the “fine print” in the above graphic reveals that the test conditions were not entirely equal. Thomson used test-time scaling — which gives it more time and computing power. Gemini and Opus ran in the more-powerful reasoning mode, but GPT-5.5 was tested in non-reasoning mode.
While non-reasoning mode is faster and more efficient for routine tasks, it is typically less capable when handling complex, advanced or multi-step challenges. That suggests that the benchmark may have understated OpenAI’s performance.
Some Benefit to Thomson
Through a Thomson Reuters spokesperson, I asked Hron about this. He said that Thomson did, in fact, benefit from test-time scaling on certain tasks, but that the overall effect was minor. “Its grand average increased from 0.787 without test-time scaling to 0.789 with it,” he said, “and the conclusions from the benchmarks would not have materially changed without it.”
With regard to the use of non-reasoning mode for GPT-5.5, Hron said that the reasoning-mode evaluation took substantially longer to complete, and that those results were not ready in time for publication.
“We expect reasoning mode to improve its performance but based on the results we have seen so far, not dramatically,” Hron said. “We are continuing to run those evaluations and plan to include the results in the technical report.”
Hron said it is also important to note that reasoning mode and test-time scaling are not necessarily direct analogs.
“A model operating in a mode described as ‘non-reasoning’ may still benefit from test-time scaling or other inference-time techniques implemented behind the provider’s API,” he said. “Because external providers generally do not disclose the full details of those systems, we cannot know precisely which techniques were applied.
“We made a best effort to ensure a fair comparison based on the information available and disclosed Thomson’s use of test-time scaling in the interest of transparency.”
The Retrieval Test
In a second evaluation, Hron and Schwarz compared Thomson’s performance when integrated with Thomson Reuters’ proprietary content with leading frontier models given unrestricted access to the web.
Specifically, they tested completeness and factuality on 53 legal research queries written by Thomson Reuters’ internal subject-matter experts. Thomson was connected via an in-house agentic harness to Westlaw and Practical Law, while the competing frontier models were given unrestricted web access through the Brave search engine.
Completeness was scored against SME-written rubrics listing every element a good answer would require. Factuality was scored by extracting each claim in a report and checking whether the cited sources actually supported it. Scoring was done by LLMs as judges, calibrated against expert scoring.
Unsurprisingly, Thomson-plus-Westlaw beat frontier-models-plus-web-search on both dimensions.
I say “unsurprisingly” because this is less a test of the models than of the retrieval sources behind them. It seems fair to assume that, for legal research questions, a model grounded in Westlaw and Practical Law should beat a model fishing the open web. It also seems that would be true regardless of whether the model was Thomson, Opus or GPT.
In that sense, the evaluation was not so much of the Thomson model itself, but of the value of Thomson Reuters’ content it drew on. What it does not tell us is what would happen if the frontier models were given the same Westlaw access.
As Anthropic, OpenAI and others push deeper into legal workflows, and as tools such as MCP connectors make it increasingly feasible to pipe authoritative legal content into general-purpose models, that is a comparison I would like to see.
Here again, I asked Hron about this. Here is his response:
“We did test other frontier models with access to Thomson Reuters content across dimensions including factuality, completeness, conciseness, understandability, relevance and coherence. Importantly, these tests did not use the CoCounsel Legal harness, which includes a more sophisticated combination of models, tools and capabilities. Instead, we tested each model using a simpler agentic harness with native search access to our content.
“Across the four models compared, overall scores ranged from 0.81 to 0.91, with Thomson scoring 0.89. This reinforces our view of the value of authoritative Thomson Reuters content, while also demonstrating that Thomson is broadly competitive with frontier models under equivalent conditions.”
Caveats and Context
Needless to say, these benchmark results are entirely self-reported on the part of Thomson Reuters, with no mention of any independent third-party verification. Undoubtedly, as Thomson is put into wider use, there will be independent evaluations.
Hron acknowledges this in a statement the company provided: “Internal evaluations are necessary because they allow us to test against the real workflows, failure modes and quality standards our customers encounter. We also recognize the importance of credible third-party validation and expect that to be an important part of how Thomson is evaluated over time.”
When I interviewed Thomson Reuters CEO Steve Hasker in June, he talked about the development of Thomson and said one of the reasons for building it was that it provided the company with a degree of flexibility beyond reliance on the flagship LLMs.
He also talked a lot about the concept the company has been emphasizing of delivering “Fiduciary-Grade AI,” and Hohn and Schwarz use that phrase twice in their post, positioning Thomson as a development that is in furtherance of that commitment.
It might also be a signal of where the legal AI market is headed.
“These results show that the most capable AI for professional work does not have to come onlyfrom the largest frontier labs,” Hron said. “Thomson is a fraction of the size and cost of many leading models, yet it performs competitively with the strongest models available and outperforms leading models in several evaluated categories. That is what becomes possible when you build the model around the work.”