@garymarcus

Some sober thinking about Mythos (full version with links at my newsletter): 1It’s probably not as bad as they say, as AI and cybersecurity expert @HeidyKhlaaf explains elsewhere (in a thread “As someone who has audited dozens of safety-critical systems, built static analysis tools, and …. here are some red flags”)
 2Whether or not Mythos is AGI per se is a red herring. (It probably isn’t; it’s telling that the report says very little about overall capabilities.) Crucially, AI doesn’t need to be AGI to cause harm! ChatGPT can’t reliably run a timer but it has still been implicated in delusions, suicides, cognitive surrender, mass disinformation, and so much more. Mythos may wreak havoc even if it hallucinates and lacks reliability outside of domains like coding and math. A system doesn’t have to be AGI to carry risks. 3The strongest lesson here is about policy. Anthropic showed some admirable restraint in not publicly releasing a potentially dangerous technology. But some of their competitors (such as OpenAI and xAI) might well not. Whether Mythos is as scary as it sounds or not, the reality is that without any government oversight on what can be released, we are entirely at the mercy of individual CEOs, some of whom have decidedly not earned our trust. 4Corollary, as pointed out to me by a friend, “it is impossible to disentangle real concerns from fear mongering being used as a marketing strategy, and so it just is not possible to separate justified panic from mere advertising which is why we need government oversight!” 5What about China? We need an international agency and treaty. That’s what I have been arguing all along. (That was the point of my 2023 TED talk, and 2023 invited essay with @AnkaReuel in The Economist). We still do. Mythos (and the reporting around Altman) only make that clearer. Self-regulation is too little, too late. The situation is this: three years of self-serving and misleading arguments about how regulation would allegedly preclude innovation has left us up shit’s creek, without a paddle. I can’t tell exactly how far Mythos (or similar competitive systems that will no doubt follow) leave us up that creek, but I do know that the time for crafting paddles is running out.

→ View original post on X — @garymarcus