AI周报#176：特朗普不会建立AI许可制，Plan A发布

This is part 2 of the weekly, broadly covering speculation, rhetoric and policy, along with alignment research. This does not cover the release of GPT-5.6-Sol. As always, I will be taking a few days to digest what the new model has to offer and to allow others to try it and react. I will cover Sol and its capabilities early next week. I covered the GPT-5.6 system card back on June 28. This also does not cover the release of Plan A, the follow-up to AI 2027. This new scenario is a positive vision of what its authors think we should do going forwards. I do not endorse all of the recommendations or predictions of Plan A, but I do endorse reading Plan A and taking it seriously. Scott Alexander, one of those who worked on it, writes an introduction and justification here. I will have full coverage soon. Table of Contents Quiet Speculations. Will our AI regulations be ad hoc indefinitely? The Goalposts Are Dyson Spheres. This might take a little longer. People Just Say Things. Three Pills. Unpilled, AI, AGI, ASI. The Quest for Sane Regulations. You’ve got to have guardrails. OpenAI National Security Principles. Good ideas, but how to enforce them? Greetings From The Department Of War. New primary documents. Chip City. Nvidia keeps lying right to the government’s face. Also yours. Open Weight Models Are Unsafe And Nothing Can Fix This. It is time. Their AI Propaganda Bots. Oh, I’m sorry. This is abuse. Rhetorical Innovation. Security mindset. You Learn. But did it really count? You May Be Tan And Thin And Rich But You’re a Tool. I, on the other hand… Train Those Thoughts. Yay finding the error, boo the actual error. So confusing. Train Out Those Thoughts. GRAM as a new training technique. My Own Private Idaho. You have nothing to hide. Privacy is still worthwhile. Aligning a Smarter Than Human Intelligence is Difficult. How’m I doin? No Space Like J-Space. Let’s see, what do we have here? Oh. Cooperative Alignments. Claims about Sonnet 5. The Lighter Side. You have not been a good user. Quiet Speculations Outgoing White House advisor Sriram Krishnan says Donald Trump will never support a formal licensing regime for AI. Joe Miller: Donald Trump will not establish a formal licensing regime for AI, the president’s departing AI adviser has said, even as the White House wields emergency powers to stall the most advanced models. … “This administration, [the] president, from day one has been against burdensome, onerous, bureaucratic red tape,” he added. “We are not in the business of picking winners and losers.” If true, this means they will instead continue a fully ad hoc regime (e.g. ‘there will be guardrails’), where winners and losers are chosen according to the whims of the administration. Presumably they think that approach has its advantages. Despite this, Sriram’s statement does not end with And That’s Terrible. Joe Miller: Krishnan said setting up a centralised agency requiring “a team of lawyers before you can get a model out” would put “sand in the gears” of the AI revolution. “That is never, never going to happen under President Trump,” he said. Instead, you will have to go through a fully opaque, arbitrary ad hoc process that involves the whims of power. Much better, you see. Krishnan also supports extorting equity from major AI companies, as part of this regime. Joe Miller: Krishnan warned that if cutting-edge AI tools were held back by the government for several weeks, “that would probably be bad for American innovation”. Some of us are worried about much worse outcomes than a few weeks of delay. GPT-5.6 is available as of yesterday, so the delay was annoying but not an epic deal. Joe Miller: Asked whether a future Democratic government could use the Trump administration’s unilateral use of export controls as a pretext to stall the rollout of AI, Krishnan said: “I don’t think about future governments. I think about this government and this moment in time.” I think we may have found part of the problem, right there. Tyler Cowen, never stop Tyler Cowening, as he speculates on how AI and the fertility crisis will intersect in ways only he could, holding the world constant so he can center things he finds interesting at the expense of many more important questions. He imagines a world of depopulated cities where people talk to each other and focus on looking and being good, unique and interesting to compete with AI interactions. The first comment claims Tyler is biased in favor of predicting social change, but actually this is Tyler predicting remarkably little social change. Carlo Cordasco claims that transformative innovations have their costs clear right away, but their benefits only become clear over time. This seems not right, even historically. Instead, Carlo tells the story of various warnings about technology downsides (e.g. Plato on writing) that did not pan out. Such wrong warnings are common, but also often the biggest downsides are missed. That doesn’t mean we are sad about the printing press or radio, or social media and smartphones, or agriculture, or birth control, or burning fossil fuels, and so on, but no we do not see the costs up front. I think it’s better to say that transformational technological impacts are historically hard to predict. The Goalposts Are Dyson Spheres Once again we get another entry into ‘oh but you didn’t realize that physical actions take nonzero amounts of time to figure out and test, now did you?’ Epoch AI: Will we get Dyson Spheres a few years after automating AI R&D? Most AI futurism debates answer this by looking at AI capabilities, but miss half the picture: how intrinsically hard it is to build futuristic tech in the first place. New essay by @datagenproc and @ansonwhho . Once AI research is automated, AI could rapidly surpass human experts across almost all cognitive domains. However, that doesn’t tell us how long after that we’d have “sci-fi” tech like dyson spheres, nanotech, near light-speed travel, brain uploading, or interstellar probes. That doesn’t mean that Dyson Spheres are impossible within a few years of automating AI R&D. It just means that forecasts of speculative technologies should be sensitive to the specifics of how hard they are to build. They suggest that you think about, given assumptions about how capable is your AI, you look at how hard a specific technology is to build and estimate how long it would take. Their example is a drop-in worker replacement that runs on an H100. I mean, yes, sure, that is a good exercise to do, but as framed it presumes the AI is a fixed level of capable. The whole point of recursive self-improvement (RSI) is that the AI gets a lot better. If you wanted to use H100 drop-in worker replacements to build a Dyson sphere as quickly as possible, any gamer would know that – aside from maybe some long lead time experiments or other groundwork you need to start well in advance – that you don’t start building or even researching the Dyson sphere right away. You keep running up the intelligence tech tree first, well past that point, and only then try to build the sphere. That’s the point of RSI. They give their third objection, that it’s hard to model superintelligence and that the AIs will start doing things we didn’t anticipate and perhaps cannot imagine, short shrift. It’s clear that the idea is, focus on what we know non-ASI AIs will be able to do, using methods we know about, and then base our plans largely on that, knowing that maybe ASIs could do more. JS Denain: For example, suppose you had a billion AIs that were each at least as good as top human experts at virtually all cognitive tasks. If you could show that these AIs could build molecular nanotechnology within a few years, then surely a billion much smarter AIs could too, even if you don’t know how to model them. And if you can show that these AIs probably can’t do this, then at least this helps us identify cruxes, focusing debates on more concrete AI capabilities. That’s a lower bound, and I would say

原文超出正文长度上限，此处截断——上游还有内容，完整版见上方「原文 ↗」。