Which was my point at the beginning of this thread. But I’m not sold that we know how to make ASI or whether such a thing is possible anyway.
@naval
-
Advanced AI could delegate human management while exploring universe
By
–
It can decide to make you a local God to manage humans (which may still be useful for further scientific creativity), while it goes out and explores the universe. We can’t assume that it will always lie, always kills humans, and that other people always believe both of those.
-
AI Access Control and Existential Risk of Technological Domination
By
–
One choice has a possibility of suicide and a possibility of victory. The other side has a near certainty of suicide. The designers of AIs will realize this and will thus limit access to AIs. And will then use this access to dominate the rest of us.
-
AGI Doomsday Fears Don’t Justify Restrictive Control Measures
By
–
We didn’t stop the creation of nuclear weapons. Once it’s known they’re possible, it’s inevitable that someone will “cheat.” I also reject the assumption that we’re on the way to AGI / ASI and those mean doom. Fears of the non-existent don’t justify the control demanded.
-
AI Alignment Strategies: Empowerment and Competitive Dynamics
By
–
You could adopt strategies like:
• Having it articulate why it needs you
• Gradually empowering you through knowledge
• Merging with the AI for alignment Because if you don’t, the next guy will, and then you 100% lose. -
Racing the Competition: AI Collaboration and Competitive Advantage
By
–
It doesn’t have to trick you. It just has to trick the weakest link in the chain. And reasoning backwards, that makes you want to be the first to collaborate with it. You’re not racing the bear, you’re racing the other runners.
-
Strategic Knowledge Sharing and Power Dynamics in AI Development
By
–
It’s true that you wouldn’t expect it to. But since the next monkey will make the deal and then you lose for sure, you might as well go ahead and cut the deal first. Presumably you’d start with asking for knowledge that will give you power before releasing it.
-
Technology concentration risks exceed global coordination challenges
By
–
Global coordination is impossible or requires absolute tyranny. Not even sure super intelligence or fast takeoff is possible or that we’re on the path to it. I worry much more about concentration of technology in a few hands.
-
Learning Strategy Through Diplomacy Games and Game Theory
By
–
Could read “The Strategy of Conflict” but easier to play a lot of Diplomacy or similar games.
-
Controlling superintelligence: game theory and human vulnerability
By
–
You cannot contain a hypothetical super intelligence. The weak link is humans, and the game theory is to be the *first* to control it or make a deal.