OpenAI and Anthropic are getting cozy with government. What could possibly go wrong?

While the world and private enterprise are adopting AI rapidly in their workflows, government isn’t far behind. The U.K. government has said early trials of AI-powered productivity tools can shave two weeks of labor off a year’s work, and AI companies are adapting to that need. More than 1,700 AI use cases have been recorded in the U.S. government, long before Elon Musk’s DOGE entered the equation and accelerated AI adoption throughout the public sector. Federal policies introduced in April on AI adoption and procurement have pushed this trend further.

It’s unsurprising that big tech companies are rolling out their own specialist models to meet that demand. Anthropic, the maker of the Claude chatbot, announced last week a series of models tailored for use by government employees. These include features such as the ability to handle classified materials and understand some of the bureaucratic language that plagues official documents. Anthropic has said its models are already deployed by agencies “at the highest level of U.S. national security, and access to these models is limited to those who operate in such classified environments.”

The announcement follows a similar one by OpenAI, the makers of ChatGPT, which released its own government-tailored AI models in January to “streamline government agencies’ access to OpenAI’s frontier models.”

But AI experts worry about governments becoming overly reliant on AI models, which can hallucinate information, inherit biases that discriminate against certain groups at scale, or steer policy in misguided directions. They also express concern over governments being locked into specific providers, who may later increase prices that taxpayers would be left to fund.

“I worry about governments using this kind of technology and relying on tech companies, and in particular, tech companies who have proven to be quite untrustworthy,” says Carissa Véliz, who researches AI ethics at the University of Oxford. She points out that the generative AI revolution so far, sparked by the November 2022 release of ChatGPT, has seen governments scrambling to retrofit rules and regulations in areas such as copyright to accommodate tech companies after they’ve bent those rules. “It just shows a power relationship there that doesn’t look good for government,” says Véliz. “Government is supposed to be the legislator, the one making the rules and enforcing the rules.”

Beyond those moral concerns, she also worries about the financial stakes involved. “There’s just a sheer dependency on a company that has financial interests, that is based in a different country, in a situation in which geopolitics is getting quite complicated,” says Véliz, explaining why countries outside the United States might hesitate to sign on to use ClaudeGov or ChatGPT Gov. It’s the same argument the U.S. uses about overreliance on TikTok, which has Chinese ties, amid fears that figures like Donald Trump could pressure U.S.-based firms to act in politically motivated ways.

OpenAI didn’t respond to Fast Company‘s request for comment. A spokesperson for Anthropic says the company is committed to transparency, citing published work on model risks, a detailed system card, and collaborations with the U.S. and U.K. governments to test AI systems.

Some fear that AI companies are securing “those big DoD bucks,” as programmer Ashe Dryden put it on Mastodon, and could perpetuate that revenue by fostering dependency on their specific models. The rollout of these models reflects broader shifts in the tech landscape that increasingly tie government, national security, and technology together. For example, defense tech firm Anduril recently raised $5 billion in a new funding round that values the company at over $30 billion.

Others have argued that the release of these government-specific models by AI companies “isn’t [about] national security. This is narrative laundering,” as one LinkedIn commenter put it. The idea is that these moves echo the norms already set by big government rather than challenging them, potentially reinforcing existing issues.

“I’ve always been a sceptic of a single supplier for IT services, and this is no exception,” says Andres Guadamuz, an AI researcher at the University of Sussex. Guadamuz believes the development of government-specific AI models is still in its early phase, and urges decision-makers to pause before signing deals. “Governments should keep their options open,” he says. “Particularly with a crowded AI market, large entities such as the government can have a better negotiating position.”

https://www.fastcompany.com/91348141/openai-and-anthropic-are-getting-cozy-with-government-what-could-possibly-go-wrong?partner=rss&utm_source=rss&utm_medium=feed&utm_campaign=rss+fastcompany&utm_content=rss

Created 2mo | Jun 10, 2025, 5:20:07 PM


Login to add comment

Other posts in this group

Is Apple getting ready to launch a PlayStation and Xbox competitor?

The Apple TV is probably my favorite device that Apple makes. While the Apple TV app is in dire need

Aug 2, 2025, 11:10:06 AM | Fast company - tech
This free Adobe tool offers Photoshop-strength background removal

Sometimes, the simplest photo feats are the most satisfying of all.

Me? I’ve lost count of the number of times I’ve needed to remove the background from an image for one reason or anothe

Aug 2, 2025, 11:10:04 AM | Fast company - tech
Google loses appeal in antitrust battle with Epic Games

A federal appeals court has upheld a jury verdict condemning Google’s Android app store as an illegal monopoly, clearing the way for a federal judge to enforce a potentially disruptive shake

Aug 1, 2025, 6:50:03 PM | Fast company - tech
Apple shares are up 2% after iPhone maker posts strong Q3 results

Apple shares rose 2% in premarket trading on Friday, after the

Aug 1, 2025, 4:30:05 PM | Fast company - tech
OpenAI pulls ChatGPT feature that showed personal chats on Google

OpenAI has removed a controversial opt-in feature that had led to some private chats appearing in Google search results, following reporting by Fast Company that found sensitive conversa

Aug 1, 2025, 2:20:02 PM | Fast company - tech
YouTube channels are being sold and repurposed to spread scams and disinformation, says new research

YouTubers dedicate their lives to building a following in hopes of creating and sustaining a livelihood. For top creators, the rewards are immense: MrBeast, the world’s biggest YouTuber, is

Aug 1, 2025, 11:50:06 AM | Fast company - tech
Tech policy could be smarter and less partisan if Congress hadn’t shut down this innovative program

Imagine if Congress had a clear-eyed guide to the technological upheavals shaping our lives. A team of in-house experts who could have flagged the risks of generative

Aug 1, 2025, 11:50:05 AM | Fast company - tech