The next wave of AI is here: Autonomous AI agents are amazing—and scary

The relentless hype around AI makes it difficult to separate the signal from the noise. So it’s understandable if you’ve tuned out recent talk about autonomous AI agents. A word of advice: Don’t. The significance of agentic AI may actually exceed the hype.  

An Autonomous AI agent can interact with the environment, make decisions, take action, and learn from the process. This represents a seismic shift in the use of AI and, accordingly, presents corresponding opportunities—and risks.

The P in GPT

To date, generative AI tools, largely subject to human supervision, have been designed to function by being pretrained (the P in GPT) on vast amounts of data such as large language models (LLMs) or other defined data sources and then to provide responses to inputs or prompts (a question or instruction) provided by users. This has proven to be an impressive way to come up with humanlike responses to queries or prompts—like a baby imitating sounds or words without really knowing what it is saying. Kind of adorable, but unlikely to conjure Newton’s Principia or a Beethoven symphony. So, are these generative tools really functioning as creative, independent beings? Doubtful. But that may be changing dramatically.

A new approach allows AI to interact directly and more autonomously with data and react in a dynamic way—a lot more like what humans do. This technology relies on autonomous AI agents, which Bill Gates believes are going to upend the software industry, bringing about the biggest revolution in computing since we went from typing commands to tapping on icons. And that may be an understatement.

AI Agents

AI agents are designed to make decisions without human intervention to perform predefined (for now) tasks. They can reach into the outside world, find data they hadn’t previously encountered, analyze it, then take action—far more like human interaction with the environment and less like relying on the fixed data universe of a chess program or a chatbot and an LLM that cannot go beyond its pretrained knowledge. Sounds great. What could possibly go wrong?

This is a major step forward, replacing a clever statistical approach to replicating human expression with something capable of taking in previously unknown outside stimuli, processing it, and taking action without having to be pretrained or retrained. We are removing our intermediate role creating and governing AI’s conceptual and decision-making universe. 

That’s both the point and the problem. It’s fair to say the AI baby is not just on its way to taking a few steps; it could be speeding down the highway in your new car, music blaring, swigging a bottle of tequila. 

The upside is clear. Less need for specific training and oversight. Scalability is only limited by compute resources. You can remove the human intermediary and send out agents to go and complete vast amounts of tasks on their own. After all, they are agents, they have agency—the ability to make decisions and choices. And mistakes.

What could possibly go wrong?

As software rather than a human actor, AI agent mistakes can be instantly and almost infinitely compounded, replicated, and cascaded. It is also a target for hackers. There are obvious doomsday scenarios like a rogue AI agent improperly triggering a massive wave of securities trading or unintentionally launching a military retaliation. When it comes to decisions with potentially catastrophic consequences, human oversight is by no means perfect, but most of us feel at least a modicum of comfort knowing there’s an expert human hand hovering over the go button.

There are less dramatic yet still highly impactful effects in the legal and compliance sphere that pose significant business risk. More and more companies are using AI-driven tools across the entire employee lifecycle, from selecting candidates to interviewing and hiring and continuing through performance assessment (raises, promotions, and termination). These tools are increasingly deploying AI agents. Providers often tout AI agents as supporting and improving the quality of critical HR decisions. But subtle errors in system design or implementation could lead to unfair outcomes. There’s a name for this phenomenon: algorithmic bias. At the same time, states are adopting laws penalizing both developers and users of such tools if their use results in unfair treatment of employees. And naturally, litigation is likely to follow.  

Risky Business

It is undeniable that AI agents present a significant opportunity to increase productivity by automating routine tasks and freeing people up for more creativity and problem-solving. But the risks are just as undeniable.

While jettisoning supervision and oversight may be a necessity with kids at a certain point, the metaphor only goes so far when it comes to the emancipation of AI through autonomous agents. For now, as we gleefully remove the training wheels, we should be mindful of balancing our understandable enthusiasm with reasonable caution to avoid any catastrophic crashes. 

https://www.fastcompany.com/91281577/autonomous-ai-agents-are-both-exciting-and-scary?partner=rss&utm_source=rss&utm_medium=feed&utm_campaign=rss+fastcompany&utm_content=rss

Created 4mo | Feb 22, 2025, 12:40:05 PM


Login to add comment

Other posts in this group

Astroworld is back in the spotlight and survivors are sharing haunting stories on TikTok

Astroworld is back in the news, and social media has some thoughts.

In November 2021, a

Jun 20, 2025, 11:10:03 PM | Fast company - tech
Your reliance on ChatGPT might be really bad for your brain

If you value critical thinking, you may want to rethink your use of ChatGPT.

As graduates

Jun 20, 2025, 6:30:02 PM | Fast company - tech
What is ‘office chair butt’? TikTok’s viral term for a real health problem

Rather than the Sunday scaries or toxic bosses, employees have unlocked a new workplace fear: office chair butt.

While not a new concern, the term has resurfaced on TikTok to describe ho

Jun 20, 2025, 4:10:07 PM | Fast company - tech
How this Parisian music streaming service is fighting AI fraud

Music streaming service Deezer said Friday that it will start flagging albums with AI-generated songs, part of its fight against

Jun 20, 2025, 4:10:06 PM | Fast company - tech
Nvidia and Hexagon’s Aeon humanoid robot brings AI-powered automation to factories

Artificial intelligence is evolving at an unprecedented pace, advancing from simple generative tasks to autonomous decision-making through

Jun 20, 2025, 4:10:05 PM | Fast company - tech
VisionOS 26 proves Apple isn’t treating the Vision Pro like a hobby

In 2023, the flagship reveal at Apple’s WWDC keynote was unquestionably the debut of

Jun 20, 2025, 1:40:08 PM | Fast company - tech
What the Wright Brothers can teach science entrepreneurs about how to survive a funding pullback

What happens when venture capital and government pull back from science entrepreneurs at the same time? Many scientists think we’re about to find out, and are looking at how we can preserve our co

Jun 20, 2025, 11:30:03 AM | Fast company - tech