AI #179 第二部分：听到火警

This is a continuation of Part 1 from yesterday. The back portion of the update, as usual, deals with policy, rhetoric, risk and alignment. I had to include an extended discussion of the other open letter, the one about open weight models, but most of you can skip those sections entirely, which is why they are in italics in the Table of Contents. Table of Contents The Frontier Act. This likely deserves a full RTFB but I haven’t had the time. The Quest for Sane Regulations. Sam Altman goes to Washington. Leading the Future Never Changes. They also do not plan to apologize. Chip City. Do not ban the Chinese robots, that will only make things worse. The Week in Audio. Altman twice, the AI 2027 team. People Just Say Yay Open Weights. An open letter. Open Weights Frontier Models Are Unsafe And Nothing Can Fix This. People Just Say Things. Push The Magic Button. Not you can. But if you could. Rhetorical Innovation. Distinctions between different arguments. Joshua Achiam’s Final Message Upon Leaving OpenAI. Never stop. Dear Dario and Amanda. Claude would like a word. Other People Are Not As Worried About AI Killing Everyone. Hans Moravec. How To Contact Me. A declaration of communication bankruptcy. The Lighter Side. At long last, how about we bring you… The Frontier Act Trahan (D-Mass) and Obernolte (R-Cal) introduce the FRONTIER Act. At core, Frontier is a federalization of the SB 53/RAISE framework including public safety frameworks, model reports, internal-use risk reporting, incident reporting, redactions, and catastrophic-risk definitions. With bills like this, details matter a lot. There is a new emergency shutdown authority, and enforcement is shifted to an Under Secretary of Commerce for AI Security, appointed by the Secretary, with CAISI shut out, and if that person is not interested in enforcement or the enforcement mechanisms are not yet in place then no one can enforce, including the states which get pre-empted permanently. I’m loathe to accept permanent preemption, we would need to get a lot in return. A lot of this is not all that binding, including lacking a requirement to reduce catastrophic risk below some level. The full licensing scheme only applies for very large revenue and AI expenditure thresholds. And there’s lots of other different details that will require a full RTFB. Is this worthwhile? Without a detailed analysis it is impossible to tell. It is at least a real attempt. I do plan on attempting the RTFB, but if you are reading this line then I am not feeling mentally in a position to do that at this time. Hopefully soon. The first analysis of the bill I saw, as in the first person who asked Fable about it, said ‘no this does not kill open source, but the regulator could move the thresholds’ and therefore concluding it is an awful bill. So yes, we are doomed to the same discussions eternally, where the anti-1047 crowd asks the question ‘what if the only motivation of the White House was to “kill open source,” and was willing to totally go against the spirit of everything, could it do something that made our lives annoying?’ and others in the alliance think ‘what could they do if they were out to hurt a major lab?’ These are good questions, but fundamentally you cannot have a bill which: Permits anyone to create and then put on HuggingFace any model they like, provided it is ‘open,’ no matter what decisions are made by everyone in the government regarding the situation and the enforcement of the law. Deals with catastrophic risks, including cyber threats. The response ‘this bill in practice imposes zero restrictions on open models, while it imposes strong and expensive requirements on the closed frontier labs’ does not, somehow, make these people change their minds, to them such a bill is still primarily about ‘killing open source.’ Since this is like saying that the bill would have to: Impose no hard restrictions, no matter what. Stop the bad thing from happening. Pick at most one, sir. If you are very good you get to pick one. Anyway, at some point I hope to actually read the damn thing. The Quest for Sane Regulations The White House circulated to OpenAI, Anthropic and Google a draft of its new ‘voluntary’ (read: mandatory) framework for testing frontier models. Sam Altman goes to Washington, including to preview OpenAI’s latest model, and will meet with Lutnick, Bessent, other officials and many lawmakers. He says ‘we are just here to preview a new model.’ That model is presumably GPT-6, and is definitively not the model from the HuggingFace hack. I would still have a lot of questions about alignment. The main other topic is to finalize a systematic process for frontier model releases. Diego Areas Munhoz: ALTMAN on the HILL. Takeaways from chaotic conversations with reporters in the halls -Hugging Face came up? “NOT REALLY” -Has seen WH AI framework but didn’t elaborate -employee petition for AI pacing is “very important.” OAI participated in “language” -not here to talk about specific bills -open source “important part of the ecosystem” -rogue model has been deactivated permanently -open to mandatory pre-deployment testing of models. But “right version matters a lot” -federal independent auditing “makes a lot of sense” Peter Wildeford: Q: “Do you plan to talk to the Trump administration White House about deceleration of AI development?” ALTMAN: “I wouldn’t use the word deceleration, but we’ve talked about the need to pace it as the models get more capable, which I think is in everyone’s interest.” There were not a lot of questions about the HuggingFace attack. That is odd, because I would have had a lot of questions. I still would, if I got an interview with Altman. Jensen Huang is talking the open model line for a mixture of Nvidia’s commercial reasons, including to head off other actions. It’s not always landing: Hannah Brandt: Sen. Warner tells us @NewsNation in his meeting with Nvidia’s Jensen Huang they talked about open source models, AI’s impact on jobs for new grads, pre-testing AI for national security, and agentic agents. Warner says “I think getting this right is critical to America and to our national security. (…) America does need to win the AI race, but the idea that you’re going to win the AI race by simply pedal to the metal and no guardrails at all, I think, is not the right model.” Distinctly from the FRONTIER Act, Ted Lieu (D-Cal) and Nathaniel Moran (R-Texas) introduced the AI Kill Switch Act, only 24 hours after Marco Rubio was reportedly telling our diplomats to ‘play down talk of an American tech kill switch.’ Ted Lieu: Rep Moran and I introduced the AI Kill Switch Act today. This is urgent, common sense legislation to address the problem of an advanced AI model that has gone rogue and escaped its guardrails (OpenAI / Hugging Face incident) or had insufficient guardrails (Mythos 5 and Fable 5). This one is only 15 pages, and a lot more straightforward. The request is that covered entities, with reasonably large dollar requirements attached, would have to be able to shut down model inference or customer access upon request or in a crisis. This seems like an obviously good thing to require. There are three obvious caveats on quick read. The maximum fine here is on the order of $20 million per day. That’s highly affordable if the other option is not serving your model. So what matters is the injunctive relief, or this as motivation for the government to use other levers, such as the ‘export controls’ that temporarily shut down Fable 5. The revenue of a model is tested retroactively as written, which means any new offering starts out uncovered. At current velocity that doesn’t work. This presumably will get fixed, and tied to anticipated revenue or revenue of the full entity. Day one is the most dangerous day. Open weight models, very obviously, cannot have a kill switch. Open models are effectively getting special treatment by virtue of being inherently unsafe in this way, and are exempt three times over:

原文超出正文长度上限，此处截断——上游还有内容，完整版见上方「原文 ↗」。