Call of Duty’s new AI moderator is listening to your voice chat

You have to be a brave soul to wander into public voice chats for modern online shooters…or the kind of jerk who just enjoys that sort of toxic environment. The Call of Duty series is infamous for its vulgar and off-putting voice chat, especially in open public games. To combat the worst of the worst players who ruin the experience for others, Activision is calling for backup from Modulate, a company that offers an AI-powered voice chat monitor designed specifically to spot bad behavior in games.

“ToxMod” is more than just a word filter, according to the blog post on CallofDuty.com (spotted by PCGamer). In addition to rather vanilla flagging of “hate speech, discriminatory language, harassment and more,” Modulate says that ToxMod can distinguish tone, intent, and context in human speech. On top of the usual four-letter words and general asshole behavior, ToxMod may be able to detect more subtle and insidious uses of in-game voice chat, like recruitment for online extremists or attempts at soliciting sexual contact with minors. ToxMod is currently being used mostly on smaller VR titles, notably Among Us VR, but CoD is its biggest client by far.

The system isn’t being given carte blanche powers, however. Speech automatically recorded and flagged by ToxMod still has to be reviewed by one of Activision’s human moderators (who probably have a really exhausting job). If the speech is found to violate Call of Duty‘s official Code of Conduct, which bars “derogatory comments based on race, gender identity or expression, sexual orientation, age, culture, faith, mental or physical abilities, or country of origin,” players will be given the appropriate moderator actions. Punishments range from a two-day suspension to full lifetime bans for repeated or particularly extreme offenses.

ToxMod is being used right now for multiplayer matches in Call of Duty: Modern Warfare II and Warzone in an initial beta, with a full rollout planned for Modern Warfare III‘s launch in November. At the moment only English chats are being monitored, but Activision plans to expand its capability to more languages at some point.