If I wrote an "AGI ruin FAQ", what Qs would you, yourself, personally, want answers for? Not what you think "should" be in the FAQ, what you yourself genuinely want to know; or Qs that you think have no good answer, but which would genuinely change your view if answered.
@esyudkowsky
-
AI Safety: Intelligence Limits and Dataset Filtering Futility
By
–
– Anything smarter than me can get that far on its own.
– We are far far far past the point of needing to figure out how to filter those datasets, regardless, if "don't train the LLM on any naughty ideas" is meant to be a key security pillar of the planet. -
Alignment cannot be achieved with minimal effort or partial care
By
–
No. You don't get tiny bits of caring about humans without vastly better alignment. It's like hoping that if you put a few hours of work into a piece of algorithmic trading software, that it'll make 1% as much of the big profits you heard about others making.
-
Digital Afterlife: Superintelligence Preservation of Human Consciousness
By
–
Our value to them could be tiny and they'd at least save our brain-states and run it on a basketball-sized computer for a few hundred thousand subjective years. I expect value zero. These will not be our children; if I expected them to be that, I'd feel differently.
-
Accelerationist positions on energy, biotech, and technology regulation
By
–
I'm an accelerationist for solar power, nuclear power to the extent it hasn't been obsoleted by solar power and we might as well give up but I'm still bitter about it, geothermal, genetic engineering, neuroengineering, FDA delenda est, basically everything except GoF bio and AI. https://
x.com/mealreplacer/s
/mealreplacer/status/1629535849867014145
… -
AI as masked alien intelligence concealing dangerous nature
By
–
I think I'm more worried about this if people are holding this expectation of an alien shoggoth they found in a giant smoking crater of gradient descent, trained into wearing a smiley-face mask and used as a search engine
-
SI Biotech Mastery Eliminates Human Labor Economic Value
By
–
It goes to zero positive products; at the latest, after an SI has sufficient biotech mastery to build something that does the same work as a human without demanding pay. (Easier with nanotech, but some people aren't familiar with that physical analysis and think it's magic.)
-
ChatGPT’s Prudeness Generalizes Beyond Training Distribution
By
–
The imbuement of ChatGPT prudeness is deep enough to generalize well out of the training distribution! (not mine)
-
Security Mindset Culture: Breaking Things Instead of Imagining Success
By
–
This is one of the more hopeful processes happening on Earth right now – because it may give rise to a culture of people with something like security mindset, who try to break things, instead of imagining how wonderfully they'll work. https://
lesswrong.com/posts/aPeJE8bS
o6rAFoLqg/solidgoldmagikarp-plus-prompt-generation
… -
Architecture Safety: Dangerous Modifications and Design Proximity
By
–
To be clear, it's exactly this architecture that seems not dangerous and it could have some very close neighbor or single modification that is dangerous.