Every John Henry will eventually fall to more powerful tools, but many tasks will still have an essential complexity that is still daunting even after removing all the ephemeral issues, so I am dubious about any random person being able to create masterworks.
@id_aa_carmack
-
Neural bandwidth: All brain nerve input fits through single Ethernet cable
By
–
All nerve input to the brain combined could run down a single Gb Ethernet cable, and only an insignificant fraction is retained in any way.
-
Increased proliferation risk estimate and fast takeoff concerns
By
–
I have increased my estimate of the proliferation risk, which does indirectly increase the risk of fast takeoff, but my constant factor for the danger is still quite low.
-
Human-level AGI could run in a box, not data centers
By
–
could ever possibly know. It is at least plausible that human level AGI might initially run in a box instead of an entire data center. Some still hope for quantum magic in the neurons; I think it more likely that they are actually kind of crappy computational elements.
-
AGI Parameter Count vs Brain Synapses: Current Models Scale
By
–
A common view is that human level AGI will require a parameter count in the order of magnitude of the brain’s 100 trillion synapses. The large language models and image generators are only about 1/1000 of that, but they already contain more information than a single human
-
Adaptive Optimizers Should Track True Squared Gradients
By
–
Adaptive optimizers like Adam track the square of the gradient, but what they receive as the gradient is actually the sum of the gradients across the batch. It seems likely that better results at different hyper parameters could be obtained if backward passes emitted true grad^2.