I think we need to back up and try to say better what this conversation is about. It's got nothing to do with human preferences. It's about AIs ending up with preferences.
ETHICS
-
Utility as Optimized Property in Outer Optimization Processes
By
–
We're not using "utility" as something that's supposed to descriptively surprisingly true of human preference; we're using it to describe properties that we expect to get optimized into a thinking process by any outer optimization that rewards good outer solutions.
-
Gary Marcus Five Critical Points for Elon Musk
By
–
sketched here: https://
garymarcus.substack.com/p/dear-elon-mu
sk-here-are-five-things
… -
Evolution’s Optimization Process and Reality’s Infinite Degrees of Freedom
By
–
Evolution's outer optimization process was aimed at "inclusive genetic fitness", a simple concept that humans never even formed except by postmortem reconstruction millions of years later. There's vastly more degrees of freedom in "where to steer reality" than "predicting
-
Humans Make Mistakes Too: LLMs vs Human Error Discussion
By
–
“but humans make mistakes, too” is so common a reply, I have written a discussion comparing humans and LLMs:
-
Groupthink versus Tulips: AI Market Rationality Questioned
By
–
https://
garymarcus.substack.com/p/groupthink-v
ersus-tulips-what-will?r=8tdk6
… -
Human Cognitive Errors vs LLM Hallucinations Explained
By
–
So many people are confused about the relation between human cognitive errors and LLM hallucinations that I wrote this short explainer: Humans say things that aren't true for many different reasons
• Sometimes they lie
• Sometimes they misremember things
• Sometimes they fail -
ASI Economic Gains and Existential Risk Scenarios
By
–
If they weren't going to kill everyone, and everyone stayed employed, competition would arise and the public would capture more of the gains from trade. It's the part where ASIs kill everyone that's a valid problem; or in a theoretically possible shorter term, a scenario where
-
Big Tech’s Defense Contractor Role and State Surveillance Provision
By
–
Extremely important research. It's not possible to understand the state of BigTech, or its pathologies at the moment, without understanding these companies' transformation into defense contractors, in addition to their central role as surveillance providers to the state.
-
Unbounded Rewards Drive General Intelligence Development
By
–
Every unbounded reward function about a complicated topic is a demand for general intelligence if you optimize hard enough. Natural selection didn't explicitly select hominids using IQ tests. It just turns out that general intelligence is a simple solution to "chip stone