Would papers get rejected if they used legal code with variable names instead of Greek laden pseudo code?
@id_aa_carmack
-
All Learning Happens Online Without Pretraining or Transfer
By
–
It’s ALL online learning in nature — no pretraining and transfer.
-
Physical Atari RL Framework: Simulation Transfer Challenges
By
–
Some interesting experiments with our “physical Atari” RL framework — one of our main points was that the flawless simulators didn’t transfer well at all to the messy reality of cameras and servos, but perhaps less obviously, even transferring from one robotroller to a second,
-
Questioning transformer sourcing delays and efficiency priorities
By
–
I have always struggled to appreciate why the big transformers take multiple years to source, and this article didn’t really clear it up. I wonder if there is a hangup on absolute efficiency akin to the US ISP fixation with rocket engines, and trading off a little bit of
-
L1 Penalty Limitations: Overshooting Zero with Standard Optimizers
By
–
L1 penalty doesn’t actually make truly sparse features with standard optimizers either — it overshoots zero back and forth.
-
Weight Decay: Soft Pruning of Noisy Features in Neural Networks
By
–
Weight decay is usually presented as “encouraging simpler solutions”, but I tend to think that the real benefit is the soft pruning of noisy / unhelpful features. Without decay, a weight can random-walk to a large value even if the input is completely random. Momentum and
-
LLM Services Security and Modern Hiring Process Shortcuts
By
–
It would have to be your account at an LLM-as-a-service, not your private LLM, or it would be obviously exploitable. You may not be aware of just how "shortcut" the hiring process is today; if you think every candidate is carefully and objectively weighed against every other
-
LLM Chat History as Extended Interview and Job Reference
By
–
In some important ways, a user’s LLM chat history is an extended interview. The social media algorithms learn what you like, but chats can learn how you think. You should be able to provide an LLM as a job reference, just like you would a coworker, manager, or professor. It can
-
Stray Cats Learning to Use Electric Blanket Like RL Agents
By
–
We have a number of stray cats in the neighborhood, and it is going to freeze in Texas tonight, so Trista put out an electric blanket for them. Watching them circle around it, only touching with one paw at a time, reminds me of RL agents in training, where you want to yell “Oh,
-
Using torch.set_default_device for CUDA Configuration
By
–
I’m fond of torch.set_default_device(‘cuda’)