A prediction problem in a discrete and finite domain is easy because we know how to represent uncertainty in the prediction.
If there are 30,000 possible tokens, we output a 30k-dimensional vector of values between 0 and 1 that sum to 1 (thank you softmax).
We can't output a
@ylecun
-
Predicting Uncertainty in Discrete Token Probability Distributions
By
–
-
Engineering Over Theory: Building Reliable Systems Without Perfect Proof
By
–
There is no math that proves that there exist a way to make safe and reliable turbojets.
Yet we do make turbojets that are incredibly safe and reliable.
It's called engineering.
You should try it. -
Meta’s Ethical Considerations for AI Product Deployment
By
–
Meta is asking itself this kind of questions literally *every* *single* *time* a product is under consideration for deployment.
-
Turbojet Safety: Engineers’ Responsibility Without External Pressure
By
–
Engineers and scientists who have made turbojets as safe and reliable as they are today haven't specifically worked on "turbojet safety," nor have they been required to "acknowledge there is a problem to be solved" by mobs of random people claiming they weren't doing their job
-
PyTorch Genesis: How Code Sharing Accelerates AI Progress
By
–
The whole genesis, history, and inspirations for PyTorch, in which @soumithchintala demonstrates that the free exchange of ideas and code makes software progress faster. Unlike with technical papers, there is no default citation mechanism for code. But citing sources of
-
Real World Challenges: Why AI Differs from Code
By
–
Language/code are easy because they are discrete domains.
Code is particularly easy because the underlying "world" is fully observable and deterministic.
The real world is continuous, high-dimensional, partially observable, and non-deterministic: a perfect storm.
The ability of -
Nuclear Weapons vs AI: Understanding Design Intent Differences
By
–
Nuclear weapons are designed to wipe out entire cities.
AI is designed to amplify human intelligence.
See the difference? -
Safe Technology Deployment Path Will Prevail Over Unsafe Alternatives
By
–
No.
I'm claiming that if there is a way to build and deploy a technology in a safe way, that's the way it will happen.
The fact that there are zillions of ways to build unsafe versions of it is completely irrelevant.
Just like with turbojets. -
AI Safety Debate: Comparing Risks to Established Technologies
By
–
BS.
1. Turbojets actually enable the destruction of entire cities by bombing. Yet we still build turbojets.
2. There is exactly zero evidence that AI can ever cause the kind of destruction you allude to. -
PyTorch Autograd Implementation and Dynamic Graphs History
By
–
PyTorch eventually incorporated a reimplementation of autograd, but that didn't happen until some time after the beginning of the project.
IIRC, @ebetica was instrumental in incorporating autograd-like features in PyTorch, thereby enabling dynamic graphs. @soumithchintala can