A study released today found ChatGPT’s link-tracking tag in 298 federal court cases across 71 courts, 133 filings with the U.S. Securities and Exchange Commission, and a host of peer-reviewed papers, government documents, and Wikipedia articles.

When ChatGPT provides a user with a hyperlink, it adds the tag, “utm_source=chatgpt.com.” Anyone paying attention can simply delete that part of the URL if they do not want the tag to appear, but it often goes unnoticed.

Last week, a company called HumanizeMy.ai, an AI writing and detection platform that also publishes research, had researchers search for the tag across six public archives and then open each document to confirm it was there.

Source: HumanizeMy.ai Research.

In the federal court archives, it found the tag in 298 cases — most often in the complaint, which is the filing that initiates a case.

Three federal districts accounted for 113 of the 298 cases: the Southern District of New York (55), the Northern District of California (31) and the Central District of California (27).

The tags were most often found in copyright cases (30), followed by other statutory causes of action, civil rights, personal injury, antitrust and patent.

The researchers conducted the search using the RECAP archive of federal court filings maintained by the Free Law Project. RECAP is not a complete record of federal filings, so the actual number of tags could be higher.

At the SEC, half of the filings that contained the tag — 67 of 133 — were in documents a company files when it asks investors for money.

One in five of those tagged links pointed to a market-research vendor, suggesting, the researchers say, that ChatGPT was used to research market size.

The researchers found the tag in 15 U.S. government documents and three U.K. government publications.

The U.S. government documents included five from the Federal Register, four records from congressional hearings, two congressional committee reports, and two federal court opinions.

Limited In What It Shows

Firat Mihci, a computational linguist and founder of HumanizeMy.ai, said the study is limited in what it shows.

It shows only that a link was copied from a ChatGPT answer, but not that any text was written by AI or that the cited source is wrong.

It also does not prove the full number of times ChatGPT was used, as anyone can easily delete the tag from the URL and leave no trace.

“The tag does not prove anyone did anything wrong,” Mihci said. “It proves nobody looked.”

Oliver Roberts, CEO and founder of Wickard.ai and co-director of the WashU Law AI Collaborative, reached a similar conclusion in an article he published earlier this year on Bloomberg Law about the ChatGPT tag. 

“This doesn’t prove misconduct, that a citation is fake, or that ChatGPT drafted the entire thing — and it doesn’t necessarily mean the cited source is unreliable,” he wrote. “But it’s a red flag.”

“At minimum, it suggests that someone likely copied a source link from ChatGPT and inserted it into a formal document without cleaning, normalizing, or perhaps independently checking the citation.”

Of course, it also tells you something even more basic: That whoever created that court or SEC filing used ChatGPT somewhere in the process. If it was trackable in 298 court cases and 133 SEC filings for no other reason than that someone failed to hit backspace a couple times, then how much more is it actually being used?

The full PDF of the report can be downloaded here.

 

Photo of Bob Ambrogi Bob Ambrogi

Bob is a lawyer, veteran legal journalist, and award-winning blogger and podcaster. In 2011, he was named to the inaugural Fastcase 50, honoring “the law’s smartest, most courageous innovators, techies, visionaries and leaders.” Earlier in his career, he was editor-in-chief of several legal publications, including The National Law Journal, and editorial director of ALM’s Litigation Services Division.