OpenAI promises greater transparency on model hallucinations and harmful content

OpenAI has launched a new web page called the safety evaluations hub to publicly share information related to things like the hallucination rates of its models. The hub will also highlight if a model produces harmful content, how well it behaves as instructed and attempted jailbreaks. 

The tech company claims this new page will provide additional transparency on OpenAI, a company that, for context, has faced multiple lawsuits alleging it illegally used copyrighted material to train its AI models. Oh, yeah, and it's worth mentioning that The New York Times claims the tech company accidentally deleted evidence in the newspaper's plagiarism case against it.

The safety evaluations hub is meant to expand on OpenAI's system cards. They only outline a development's safety measures at launch, whereas the hub should provide ongoing updates. 

"As the science of AI evaluation evolves, we aim to share our progress on developing more scalable ways to measure model capability and safety," OpenAI states in its announcement. "By sharing a subset of our safety evaluation results here, we hope this will not only make it easier to understand the safety performance of OpenAI systems over time, but also support community efforts⁠ to increase transparency across the field." OpenAI adds that its working to have more proactive communication in this area throughout the company. 

Interested parties can look at each of the hub's sections and see information on relevant models, such as GPT-4.1 through 4.5. OpenAI notes that the information provided in this hub is only a "snapshot" and that interested parties should look at its system cards. assessments and other releases for further details. 

One of the big buts to the entire safety evaluation hub is that OpenAI is the entity doing these tests and choosing what information to share publicly. As a result, there isn't any way to guarantee that the company will share all its issues or concerns with the public.

This article originally appeared on Engadget at https://www.engadget.com/ai/openai-promises-greater-transparency-on-model-hallucinations-and-harmful-content-184545691.html?src=rss https://www.engadget.com/ai/openai-promises-greater-transparency-on-model-hallucinations-and-harmful-content-184545691.html?src=rss