How AI is helping an Israeli TV reporter speak clearly again

When a renowned Israeli TV journalist lost his ability to speak clearly because of ALS, he thought his career might be over. But now, using artificial-intelligence software that can recreate his widely recognized gravelly voice, Moshe Nussbaum — known to generations of viewers simply as “Nussi” — is making a comeback.

Nussbaum, 71, was diagnosed two years ago with amyotrophic lateral sclerosis, a progressive disease also known as Lou Gehrig’s disease that attacks nerve cells that control muscles throughout the body.

At the time, he vowed to viewers of Israel’s Channel 12 News to continue working as long as he was physically able. But, gradually, it became more and more difficult.

It was a devastating blow to the career of a leading, no-nonsense reporter who for more than 40 years had covered many of Israel’s most important stories from the field. He had appeared from the scenes of suicide bombing attacks and the front lines of wars in Gaza and Lebanon, and had covered scandals in Israel’s parliament and high-profile court cases.

After Hamas’ Oct. 7, 2023, attack that triggered the war in Gaza, Nussbaum was unable to report from the field. It was the first war of his career he had ever sat out, he noted in a recent interview with colleagues at Channel 12, the country’s largest station.

Even though he was having trouble moving and speaking, he launched a segment interviewing injured soldiers from Israeli hospitals. His questions were slow and halting, but he kept it up for the first half of the war. Then, as it became increasingly difficult to speak, and to be understood, his interviews became less frequent.

On Monday, Channel 12 made the surprising announcement that it would bring Nussbaum back on the air in the coming weeks as a commentator— with the help of AI.

“It took me a few moments to absorb it and to understand that it is me speaking now,” Nussbaum told the Associated Press via text message. “Slowly, slowly, I’m understanding the incredible meaning of this device for everyone with disabilities, including me.”

Nussbaum will report his stories, and then write them up, using an AI program that has been trained to speak using Nussbaum’s voice. He will be filmed as if he were presenting, and his lips will be “technologically adjusted” to match the words.

People with speech disorders have used traditional text-to-speech technology for years, but those voices sound robotic and flat, and lack emotion. In contrast, AI technology is trained using recordings of a person’s voice — there are thousands of hours of Nussbaum speaking thanks to his lengthy career in TV and radio — and it can mimic their intonations and phrasing.

Thrilled by the possibilities the technology affords him, Nussbaum said he is also worried about the ease with which the technology could be used by bad actors to spread fake news and falsehoods.

In its current form, the technology will not work for live broadcasts, so Nussbaum won’t be able to go out into the field, which is his favorite part of the job, he said. Instead, he will focus on commentary and analysis about crime and national security, his areas of expertise for decades.

Ahead of the broadcasts, Channel 12 released a preview showing snippets of Nussbaum speaking naturally — garbled and difficult to understand — followed by the new “Nussi AI.” The new version sounds strikingly like the old Nussbaum, speaking quickly and emphatically. Nussbaum was filmed as if he was presenting the report, sitting straight with his trademark bushy eyebrows moving up and down for emphasis.

“Honestly, this is my first time sitting here in the studio after more than a year,” AI Nussbaum says in the preview. “It feels a bit strange, and mostly, it tugs my heart.”

AI-powered voice cloning has grown exponentially in recent years. Experts have warned that the technology can amplify phone scams, disrupt democratic elections and violate the dignity of people — living or dead — who never consented to having their voice recreated to say things they never said.

It’s been used to produce deepfake robocalls mimicking President Joe Biden. In the U.S., authorities recently charged a high school athletic director with using AI to generate a fake audio clip of the school’s principal making racist remarks.

But the technology also has tremendous potential to help people who have lost their ability to speak clearly. A U.S. congresswoman who cannot speak due to complications from Parkinson’s and a related palsy has used a similar AI program to give a speech on the House floor, and the technology has also helped a young woman who lost her voice due to a tumor.

Channel 12 declined to say which AI program it was using.

Nussbaum had worried that ALS would rob him of the career he loved. In an interview with Channel 12, he recounted telling his managers “don’t feel like you’re pitying me, doing me a favor,” he said. “The day you come to the conclusion that this is it — tell me. I’ll know how to accept it without a problem.”

He calls his new AI-enabled persona a “magic trick” that enabled his comeback, and believes it will raise awareness in Israel of ways that people with disabilities — especially progressive disabilities — can continue to work.

“The fact that Channel 12 and my news managers are allowing me to reinvent myself anew, that is one of the most important medicines I can get in my fight with this disease,” he said.

—Melanie Lidman, Associated Press

https://www.fastcompany.com/91256749/ai-nussbaum-israeli-tv-reporter-speak-again?partner=rss&utm_source=rss&utm_medium=feed&utm_campaign=rss+fastcompany&utm_content=rss