AI Chatbots have telltale quirks. Researchers can spot them with 97% accuracy

There’s a cat-and-mouse game between those using generative AI chatbots to produce text undetected and those trying to catch them. Many believe they know the telltale signs—though as a journalist fond of the word “delve” and prone to em-dashes, I’m not so sure.

Researchers at four U.S. universities, however, have taken a more rigorous approach, identifying linguistic fingerprints that reveal which large language model (LLM) produced a given text.

“All these chatbots are coming out every day, and we interact with them, but we don’t really understand the differences between them,” says Mingjie Sun, a researcher at Carnegie Mellon University and lead author of the study, which was published in Cornell University’s preprint server arXiv. “By training a machine learning classifier to do this task, and by looking at the performance of that classifier, we can then assess the difference between different LLMs.”

Sun and his colleagues developed a machine learning model that analyzed the outputs of five popular LLMs, and was able to distinguish between them with 97.1% accuracy. Their machine learning model uncovered distinct verbal quirks unique to each LLM.

ChatGPT’s GPT-4o model, for instance, tends to use “utilize” more than other models. DeepSeek is partial to saying “certainly.” Google’s Gemini often prefaces its conclusions with the word “essentially,” while Anthropic’s Claude overuses phrases like “according to” and “according to the text” when citing its sources.

xAI’s Grok stands out as more discursive and didactic, often reminding users to “remember” key points while guiding them through arguments with “not only” and “but also.”

“The writing, the word choices, the formatting are all different,” says Yida Yin, a researcher at the University of California, Berkeley, and a coauthor of the paper.

These insights can help users select the best model for specific writing tasks—or aid those trying to catch AI-generated text masquerading as human work. So, remember: according to this study, if a model utilizes certain words, it’s certainly possible to identify it.

https://www.fastcompany.com/91286162/ai-chatbots-have-telltale-quirks-researchers-can-spot-them-with-97-accuracy?partner=rss&utm_source=rss&utm_medium=feed&utm_campaign=rss+fastcompany&utm_content=rss

созданный 2mo | 3 мар. 2025 г., 14:30:07


Войдите, чтобы добавить комментарий

Другие сообщения в этой группе

An Arizona family used AI to recreate a road rage victim’s voice

The family of a man killed in a 2021 road rage incident in Arizona used artificial intelligence to portray the victim delivering his own impact statement during his killer’s sentencing hearing, ac

6 мая 2025 г., 22:40:04 | Fast company - tech
Justice Department asks court to break up Google’s ad tech business

The U.S. Justice Department is doubling down on its attempt to break up Google by asking a federal judg

6 мая 2025 г., 20:30:03 | Fast company - tech
OpenAI’s nonprofit mission fades further into the rearview

OpenAI was founded as a nonprofit with a mission to build safe artificial general intelligence for the be

6 мая 2025 г., 20:30:02 | Fast company - tech
DoorDash agrees to buy Deliveroo for $3.9 billion

DoorDash, the ubiquitous U.S. food delivery app, has

6 мая 2025 г., 18:10:05 | Fast company - tech
How Trump’s meme coin business is profiting off a dinner with the president

In the crypto world, meme coins are mostly just jokes with no intrinsic value. But the Trump family is p

6 мая 2025 г., 18:10:04 | Fast company - tech
Elon Musk faces backlash from Austin neighbors over his ‘Fort Knox’-style mansion

Elon Musk has ruffled plenty of feathers since stepping more prominently into the political arena. Now, it seems he’s also a bad neighbor.

In 2022, Musk purchased a six-bedroom, $6 milli

6 мая 2025 г., 18:10:04 | Fast company - tech
Why selling your business isn’t the only path to success

Prioritizing growth to sell is a perfectly reasonable business strategy. Being acquired by a larger group at some point (like

6 мая 2025 г., 15:40:08 | Fast company - tech