Military AI is here. Some experts are worried

For the first two-and-a-half years of the generative AI revolution, the AI arms race has been waged between competing companies seeking to make bank from the promise and potential of the technology. But things are maturing in the AI world—and with it, there’s another frontline for AI: the military.

Scale AI, the company set up by Alexandr Wang, has been awarded what CNBC reports is a multimillion-dollar deal to help develop Thunderforge, which the U.S. Department of Defense calls “an initiative designed to integrate artificial intelligence into military operational and theater-level planning, and fusing cutting-edge modeling and simulation tools.” Wang told CNBC that “our AI solutions will transform today’s military operating process and modernize American defense” and that they “will provide our nation’s military leaders with the greatest technological advantage.”

The move is unsurprising—militaries are always keen on keeping at the cutting edge of technology, trying to eke out an advantage against competitive armies—but disappointing, says Margaret Mitchell, researcher and chief ethics scientist at Hugging Face, an AI company. “We already know we’re moving forward to push AI systems farther and farther out from our control,” she says. “Many in the industry and in the media are treating more and more powerful systems as if they are inevitable, and therefore making it so.”

Mitchell adds that technology has always relied on military clientele to act as a crucible for, and accelerant of, new innovation. “Military use has long been a staple of technological development,” she says. “Massively destructive outcomes are fully predictable based on history and how the tech marketplace works.” (Scale AI declined to comment. The Defense Department did not respond to Fast Company‘s request for comment.)

That level of destruction could be catastrophic, argues David Krueger, assistant professor at the University of Montreal, studying AI safety and risk. “I think it’s likely to lead to the end of humanity, to human extinction,” he says, speaking generally about the use of AI for military purposes, calling the military use of AI “one of the most obvious ways in which AI poses an existential risk to humanity.” 

Krueger says that AI is being used in many areas to hand off human control and outsource it to AI systems. “I think this is a risk in every domain, and I think in the military, it’s particularly concerning, and something which will require international collaboration to avoid getting out of hand and risking human extinction.” Scale AI has said that the Thunderforge program will operate with human oversight, and Noah Sylvia, a research analyst at the Royal United Services Institute (RUSI), points out that “as AI functions go, I would say it is not as controversial as a lot of other ones, because this is what you could term an enterprise function.”

Scale AI is far from the only company to ink a deal with the U.S. military to leverage the power of AI to support such activities. A number of companies have also agreed terms to provide their AI technology for military purposes. “I think part of the reaction is because they started out in a very civilian-oriented company, and over the past few months, especially, we’ve seen all of these civilian companies suddenly turn towards defense more,” says Sylvia. Indeed, the press release by the Defense Innovation Unit announcing Scale AI’s deal for Thunderforge points out the same program will also include Anduril’s Lattice software platform and state of the art LLMs enabled by Microsoft.

“I struggle to see a way out of it,” says Hugging Face’s Mitchell. Even if individual countries or companies were to decide to step aside from using AI for military purposes, or to decline to provide support to countries that are seeking military AI—as Hugging Face has refused to do in the past—others would likely step into the breach. “We need some ability to coordinate to prevent actors from building AI systems,” says the University of Montreal’s Krueger. “I think that should be—in fact—the number-one priority in foreign policy for every country at this point because it’s an incredibly important issue, and it’s going to be difficult to address it.”

Developing cross-country guidelines for how to consider the use of AI in military environments will be vital in the future, says Mitchell. She suggests a multipoint plan that includes keeping AI systems within strict operational boundaries, making it impossible for systems to autonomously deploy weapons, introducing safety mechanisms, and advancing what’s deemed state of the art in input data analysis and output evaluation to gain a deeper understanding into what systems can and cannot do.

She also has two simpler suggestions. “Do not deploy technology whose actions you cannot reasonably foresee,” she says. And secondly: “Do not fully cede human control.”

https://www.fastcompany.com/91295381/military-ai-is-here-some-experts-are-worried?partner=rss&utm_source=rss&utm_medium=feed&utm_campaign=rss+fastcompany&utm_content=rss

Létrehozva 6mo | 2025. márc. 12. 12:10:04


Jelentkezéshez jelentkezzen be

EGYÉB POSTS Ebben a csoportban

This viral grocery hack will help you save money and reduce waste

If you dread the weekly grocery shop, or get sidetracked by fun snacks only to end up with no real meals, this might be the hack for you.

The 5-4-3-2-1 method gives shoppers like you a s

2025. aug. 31. 13:10:02 | Fast company - tech
Do Trump’s tariffs mean you’ll pay more for the iPhone 17 next month?

If 2025 is the year of anything, it is the year of the tariff. Ever since President Trump unleashed his

2025. aug. 30. 11:30:07 | Fast company - tech
This simple free service makes sharing PDFs painless

Look, I’m not gonna lie to ya’: I’ve got a bit of a love-hate relationship with PDFs. And, more often than not, it veers mostly toward the “hate” side of that spectrum.

Don’t get m

2025. aug. 30. 11:30:04 | Fast company - tech
Palantir is mapping government data. What it means for governance

When the U.S. government signs contracts with private technology companies, the fine print rarely reaches the public. Palantir Technologies, however, has at

2025. aug. 30. 9:10:09 | Fast company - tech
‘The New York Times’ paywalled the Mini Crossword and the internet is in shambles

Bad news for morning routines everywhere: The New York Times has put its Mini Crossword behind a paywall.

On Tuesday, instead of their usual puzzle, players were met with a paywall. The

2025. aug. 29. 19:20:05 | Fast company - tech
Chinese tech giant Alibaba aims to fill Nvidia void with its new AI chip

China’s Alibaba has developed a new chip that is more versatile than its older chips and is meant to serve a broader range of

2025. aug. 29. 16:50:06 | Fast company - tech
How Japan is using AI to prepare Tokyo residents for a Mount Fuji volcanic eruption

Mount Fuji hasn’t erupted since 1707. But for Volcanic Disaster Preparedness Day, Japanes

2025. aug. 29. 14:40:03 | Fast company - tech