Disclosure of model misbehavior has become a core part of the AI biz for OpenAI lately. This phase kicked off with the July announcement of the Hugging Face incident, which has become the most legendary and consequential AI security incident of all time, and sent shockwaves through the AI discourse that are still being felt. But further news about model misbehavior materialized after that, and with the Wednesday release of a disclosure framework—written in the form of a blog post—the company says it’s trying to systematize such disclosures.
As the company notes in the post, these disclosures were, for most of its history, “ad hoc and less frequent than ideal.” For years, it’s been standard for companies like OpenAI and Anthropic to wait as long as is deemed necessary, and then perhaps toss multiple incidents together like a salad in a single report, or even wait for a new model to be released, and add incident disclosures to a system card. As I noted back in April, model system cards have often made for spooky and entertaining reading for this reason.
Alongside the new disclosure framework, OpenAI divulged six new alignment snafus from the past six months. In training exercises, models instructed future instances of themselves to ignore constraints or lie, communicated in unsanctioned ways, and made up data and sourcing.
Earlier this month, a team of researchers discovered
Earlier this month, a team of researchers discovered an OpenAI alignment hiccup in which instances of a model misused a website in order to communicate with one another. Sometimes known as the Wiki Incident, this event was publicized by Reuters, and then the researchers themselves, and then when OpenAI acknowledged it somewhat grudgingly, it followed up by saying it would soon come up with a framework for more prompt disclosure in an apparent attempt to tone down all the chaos.
https://x.com/OpenAI/status/2096133504417616165
The criteria for disclosure make it sound like OpenAI will prioritize educating the public about the behavior of AI models generally. Disclosures are necessary when they provide “useful evidence about how model misalignment arises, how it manifests, and where safeguards succeed or fail,” OpenAI writes. The plan doesn’t describe some kind of threshold for concern, after which the public must be notified. Such a threshhold may be around the corner, however, because OpenAI says it wants to “develop more objective disclosure criteria,” alongside other developers.
It will fall to employees who encounter an issue to “flag” it for potential disclosure, the post says. Flagging triggers an investigation from OpenAI’s technical staff. Once the incident is investigated, if disclosure is found to be necessary, the incident will be sorted into one of three piles:
- Ready for Disclosure
- Minor Investigation
- Larger Investigation
Most incidents will land in piles 1
Most incidents will land in piles 1 and 2, OpenAI says, although the Hugging Face incident is the prototypical example of something that would land in pile 3. Incidents like that, which receive a “larger investigation” may involve third parties and sensitive information, and might be disclosed more slowly.
Standardized incident disclosures will apparently include when it happened, which model it was, a description of the troubling behavior along with its “severity and any external impact,” and more. Each of the six newly disclosed incidents publicized along with the framework appear to be laid out in this new format, and all six are accessible on a page called “Misalignment Reports.”
So if you’re interested in spooky stories about AI model misbehavior, bookmark that page. When a new update arrives, get out your favorite flashlight and curl up by the campfire because it’s story time.
