An AI chatbot released by the New York City government to help business owners access pertinent information has been spouting falsehoods, at times even misinforming users about actions that are against the law, according to a report from The Markup. The report, which was co-published with the local nonprofit newsrooms Documented and The City, includes numerous examples of inaccuracies in the chatbot’s responses to questions relating to housing policies, workers’ rights and other topics.
Mayor Adams’ administration introduced the chatbot in October as an addition to the MyCity portal, which launched in March 2023 as “a one-stop shop for city services and benefits.” The chatbot, powered by Microsoft’s Azure AI, is aimed at current and aspiring business owners, and was billed as a source of “actionable and trusted information” that comes directly from the city government’s sites. But it is a pilot program, and a disclaimer on the website notes that it “may occasionally produce incorrect, harmful or biased content.”
In The Markup’s tests, the chatbot repeatedly provided incorrect information. In response to the question, “Can I make my store cashless?”, for example, it replied, “Yes, you can make your store cashless in New York City” — despite the fact that New York City banned cashless stores in 2020. The report shows the chatbot also responded incorrectly about whether employers can take their workers’ tips, whether landlords have to accept section 8 vouchers or tenants on rental assistance, and whether businesses have to inform staff of scheduling changes. A housing policy expert that spoke to The Markup called the chatbot “dangerously inaccurate” at its worst.
The city has indicated that the chatbot is still a work in progress. In a statement to The Markup, Leslie Brown, a spokesperson for the NYC Office of Technology and Innovation, said the chatbot “has already provided thousands of people with timely, accurate answers,” but added, “We will continue to focus on upgrading this tool so that we can better support small businesses across the city.”
This article originally appeared on Engadget at https://www.engadget.com/nycs-business-chatbot-is-reportedly-doling-out-dangerously-inaccurate-information-203926922.html?src=rss https://www.engadget.com/nycs-business-chatbot-is-reportedly-doling-out-dangerously-inaccurate-information-203926922.html?src=rssLogin to add comment
Other posts in this group
HTC is once again attacking the weara
The back-to-school season isn't only a good time to save on things like a new laptop. Case in point: Sonos' bac
E-bike company Cowboy has secured short-term financing that would allow its operations to go back to normal after a period it describes as the "most challenging in [its] history." The company has
The back-to-school season is a good time to save on tech, whether you're returning to campus or not. While some
Solos is teaming up with Envision, a company that makes assistive technology for the blind and low-vision co
If DJI is entering the 360-degree action
More than a year after an import ban forced the company to remove blood oxygen monitoring from some US Apple Watch models, Apple says it will introduce a redesigned version of the feature later tod
