Well, that is objectively not what the models do, so they are just wrong. The question of the net social benefits of automating skilled craft is open for discussion. Generally, I think “Learn from all that you see” is virtuous behavior, and I would have a hard time condemning it.
@id_aa_carmack
-
Google Cloud GPU Allocation Issues: Company Struggles to Purchase Resources
By
–
A while ago, I had a ridiculous time trying to get a bunch of GPUs on Google Cloud. Complaining on Twitter got it resolved for me then, but we just went through it again for the Keen corporate account. Something is wrong when it is so hard to get a company to take your money.
-
Max-Pooling as Spatial ReLU: A Reframing of Network Operations
By
–
I started out with an instinctive dislike of max-pooling in ML networks, favoring avg-pooling based on image filtering concepts. I only later realized that it isn’t really doing image processing, but rather acting as a “spatial ReLU”.
-
Tools Progress Cannot Eliminate Essential Complexity in Masterwork Creation
By
–
Every John Henry will eventually fall to more powerful tools, but many tasks will still have an essential complexity that is still daunting even after removing all the ephemeral issues, so I am dubious about any random person being able to create masterworks.
-
Neural bandwidth: All brain nerve input fits through single Ethernet cable
By
–
All nerve input to the brain combined could run down a single Gb Ethernet cable, and only an insignificant fraction is retained in any way.
-
Increased proliferation risk estimate and fast takeoff concerns
By
–
I have increased my estimate of the proliferation risk, which does indirectly increase the risk of fast takeoff, but my constant factor for the danger is still quite low.
-
Human-level AGI could run in a box, not data centers
By
–
could ever possibly know. It is at least plausible that human level AGI might initially run in a box instead of an entire data center. Some still hope for quantum magic in the neurons; I think it more likely that they are actually kind of crappy computational elements.
-
AGI Parameter Count vs Brain Synapses: Current Models Scale
By
–
A common view is that human level AGI will require a parameter count in the order of magnitude of the brain’s 100 trillion synapses. The large language models and image generators are only about 1/1000 of that, but they already contain more information than a single human
-
Adaptive Optimizers Should Track True Squared Gradients
By
–
Adaptive optimizers like Adam track the square of the gradient, but what they receive as the gradient is actually the sum of the gradients across the batch. It seems likely that better results at different hyper parameters could be obtained if backward passes emitted true grad^2.