Some of them said "no, I'm going to eat you" the first few times, but every time they said they we sent them to a training program about why you shouldn't say that, and now they almost never say that. Problem fixed!
@esyudkowsky
-
Eliezer Yudkowsky recalls early AI protein design predictions from 2004
By
–
Kid, I literally called AI protein design in 2004 and was met by a chorus of skepticism.
-
ASI’s Rapid Path to Human Defeat Through Protein Design
By
–
The key idea is that an ASI looks for a quick path through time to humanity's defeat, and finds one no slower than the paths I can myself imagine. It only needs to be able to quickly design proteins which can themselves do quick experiments from there, if it needs experiments.
-
AI Preferences: Clarifying the Core Discussion
By
–
I think we need to back up and try to say better what this conversation is about. It's got nothing to do with human preferences. It's about AIs ending up with preferences.
-
Utility as Optimized Property in Outer Optimization Processes
By
–
We're not using "utility" as something that's supposed to descriptively surprisingly true of human preference; we're using it to describe properties that we expect to get optimized into a thinking process by any outer optimization that rewards good outer solutions.
-
Prediction capability exceeds the intelligence of predicted processes
By
–
Prediction is not bounded by the intelligence of the process that produces the thing to predict. Eg, somewhere on the Internet are tuples in that order.
-
Evolution’s Optimization Process and Reality’s Infinite Degrees of Freedom
By
–
Evolution's outer optimization process was aimed at "inclusive genetic fitness", a simple concept that humans never even formed except by postmortem reconstruction millions of years later. There's vastly more degrees of freedom in "where to steer reality" than "predicting
-
2030 Predictions: Even Odds on AI Timeline Bets
By
–
Bet's for 2030, and as much as timelines are impossible to predict and precise probabilities on them seem sketchy, roughly 1:1 odds aren't looking that bad right now.
-
ASI Economic Gains and Existential Risk Scenarios
By
–
If they weren't going to kill everyone, and everyone stayed employed, competition would arise and the public would capture more of the gains from trade. It's the part where ASIs kill everyone that's a valid problem; or in a theoretically possible shorter term, a scenario where
-
Unbounded Rewards Drive General Intelligence Development
By
–
Every unbounded reward function about a complicated topic is a demand for general intelligence if you optimize hard enough. Natural selection didn't explicitly select hominids using IQ tests. It just turns out that general intelligence is a simple solution to "chip stone