OpenAI解雇三名安全研究员，引发内部信任危机

Three OpenAI safety researchers, Tomek Korbak, Mikita Balesni and Jasmine Wang, were fired, ostensibly for ‘mishandling sensitive company information.’

Those employees have published an open letter. The employees have said they believe they were fired for safety advocacy, and that the reasons given were pretextual.

The letter calls upon OpenAI to take actions going forward on open discussion and safety auditing, which OpenAI says they strongly support.

OpenAI responded, saying there was an additional breach of trust that was not mentioned in the employee statements, without claiming that the employees were (or were not) informed of what it was. As you would expect given OpenAI’s track record, most outside of OpenAI who discussed this believe the researchers and not OpenAI. Whereas those at OpenAI have not offered comment at all.

No matter what happened, and how this is handled from here, this sucks bigly. OpenAI has three less good safety researchers. Trust will be lower. Speech will be chilled.

Table of Contents

• You’re Fired.
• The Initial Statement from OpenAI.
• Earned Skepticism.
• Some Highly Reasonable Requests.
• Chilling Effect.
• OpenAI Firing These Employees For Safety Advocacy Would Be Deeply Stupid.
• Why Were They Fired?
• Tomek Korbak.
• Mikita Balesni.
• Jasmine Wang.
• Operational Security Alert.
• OpenAI Responds.
• A Significant Breach of Trust.
• No One Is Buying OpenAI’s Non-Explanations.

You’re Fired

All three had expressed alarm recently on Twitter:

Jasmine Wang: It’s hard to overstate how dangerous speeding towards RSI is.

That’s why I + 1385 others signed the pacing the frontier petition asking the US government to pace AI development. I’m guesstimating this is ~8-10% of all frontier lab employees.

Mikita Balesni: I am at OpenAI and i think AI is >10% likely to kill all humans

this proposal [to avoid neuralese] is among the top things we should do as an industry to lower that risk (it’s not enough though!)

Tomek Korbak: I’m quite unhappy with much of what OpenAI does.

I am very happy that I’m allowed to say “I’m quite unhappy with much of what OpenAI does.”

These are legit researchers who were at the center of safety efforts. OpenAI firing them is a tragedy, whether or not OpenAI was justified in firing them, or was effectively forced to do so. Tomek Korbak in particular had also been one of the louder OpenAI voices of concern about recent alignment difficulties with Astra and its lack of monitorability.

Nathan Calvin: Tomek and Mikita were the lead authors of the extremely influential Chain of Thought Monitorability paper. Them leaving OpenAI (and if the WSJ story is referring to them, not by their own volition) right when safety monitorability is collapsing is terrible.

Sneha: When I think about the biggest OpenAI safety developments of the last few months, a few things come to mind: “pacing the frontier”, the METR-Redwood Hugging Face investigation, and ongoing efforts to push a cross-lab monitorability standard. Jasmine, Tomek, and Mikita were central to literally every single one of those things.

From what I can tell, they pushed their company to be better internally and defended their company when they thought external criticisms were unfair or overblown.

Their impact goes way beyond OpenAI alone. I think the public + definitely all current OpenAI staff deserve an ironclad case for why the company had no choice but to fire these three vital safety employees in one go.

The Initial Statement from OpenAI

We were given an official statement, but it did not say much.

Keach Hagey, Maxwell Zeff and Berber Jin (WSJ): OpenAI has parted ways with three researchers for allegedly sharing confidential company information with a third-party AI-safety organization, according to people familiar with the matter.

… “We have parted ways with three individuals for violating our policies on accessing and handling sensitive company information,” said an OpenAI spokesperson in a statement. “Our investigation confirmed that these individuals mishandled sensitive information outside established company procedures, violating our policies and breaking the trust essential to our work.”

OpenAI also said that the misconduct went past sharing information with an outside analysis firm, and that no one was fired for raising safety concerns. They declined at the time to name the fired employees. ​

We don’t know, from that description, where this lands from ‘fired the whistleblowers on a pretext’ to ‘holy hell, you cannot do that, yes we actually have to fire you.’ There is mishandling information, and then there is mishandling information.

We still don’t have a public explanation beyond OpenAI’s initial statement. There are many reasons that could be the case. Some are benign. Others are not.

Earned Skepticism

It was very reasonable of many to assume until proven otherwise, given the track record of OpenAI and the lack of further explanation, that it was that first one.

Remember that Leopold Aschenbrenner credibly claims he was fired from OpenAI on a pretext for taking security concerns to the board, and there are many other cases that at least rhyme with the current one.

The incident with Leopold might be impacting this one. Leopold says he was told, very explicitly, the action that got him fired. That made OpenAI look worse, not better.

Pamela Mishkin (Formerly OpenAI): If you’re at OpenAI (or any lab!), I hope you’re thinking carefully about what you work on now. You can be fired any moment and the company can reneg on its commitments around where/how your research is deployed.

Now is a great time to pace. Take a long lunch, call your mom, call me!

Systems like embedded evaluators actually working depend on safety researchers having clear lines of trust with exec and those evaluators.

These firings are a clear effort to undermine the actual efficacy of the “commitments” labs have made and have a chilling effect on any employee dissent.

girish sastry (Formerly OpenAI): OpenAI has been here many times. It throws around vague explanations, expecting the story to fade from institutional and public memory.

We will never have perfect information about situations like this. Once is happenstance, twice is coincidence, a zillion times is… OpenAI.

Some Highly Reasonable Requests

We learned from a WSJ report that the three fired OpenAI employees were calling for additional safety measures around preserving chain-of-thought monitoring, and that they (as one would expect) take issue with OpenAI’s characterization of the firings:

Maxwell Zeff (WSJ):

Three fired OpenAI employees called on the company to work with outside safety auditors and to preserve the ability to monitor increasingly sophisticated AI models as it grapples with how to best manage the novel risks of its technology.

In a letter addressed to OpenAI board members and safety committees, which was reviewed by The Wall Street Journal, the fired employees wrote that they were concerned artificial-intelligence companies could end up losing the ability to monitor AI systems’ chain-of-thought.

We now have the full letter.

Here are their parting recommendations to OpenAI:

• OpenAI must adhere to last month’s public commitments to embed third-party
• safety auditors within the organization. Our firing should not be used as a pretext for stepping away from those partnerships.
• OpenAI must preserve the monitorability of frontier models.
• OpenAI must continue to support an open and transparent culture of dialogue
• between its safety researchers and the rest of the safety ecosystem.

They go into more detail in the full letter. Everyone I respect on these matters fully agrees with these recommendations.

So does OpenAI, or at least they are strongly saying that they agree.