OTOH it's foundational skills you learn just once, and can then re-use for the rest of your life. I put in the time to learn them 30 years ago, and it's still paying off every day today. 🙂
@jeremyphoward
-
Think Python book resource for learning programming
By
–
And this book too https://
allendowney.github.io/ThinkPython/ -
Fast.ai Course Platform Built Entirely on Jupyter Notebooks
By
–
And the whole of http://
course.fast.ai is notebooks. -
Graph Breaks in AI Training: Python Runtime Considerations
By
–
Graph breaks are a fact of life, and that's fine. You can't really expect your entire end to end training to never touch the python runtime. It doesn't mean that you should try to somehow reinvent Python compilation from scratch however.
-
PyTorch Team Shows Efficient Llama Training Methods
By
–
Yes I know. We helped show the PyTorch team how to train Llama efficiently – they have a whole GPU Mode talk about it. These issues do not at all line with your claims however.
-
Understanding torch.compile: Common misconceptions explained
By
–
Whoever wrote that seems to not understand fundamentally what torch.compile does, or why it does it.
-
DDP Gradient Bucketing and Activation Checkpointing Optimization
By
–
I'm sorry, this makes no sense.
– Gradient bucketing is literally part of DDP
– DDP is a networking thing, obs you don't compile that
– Activation checkpointing works fine once hooks are disabled during tracing (ofc) -
Understanding the Inevitable Problem with AI and Technology
By
–
Everyone sees the problem eventually. They often slam into it at great speed, in fact…
-
Claude Uses ULMFiT Tokenization Technique from 2017
By
–
According to this research it sounds like Claude uses the same tokenization trick I invented for ULMFiT in 2017! 😀
-
Paywall llms.txt feature enhances premium customer experience
By
–
This is kinda cool actually — an llms.txt behind a paywall; very nice extra touch to provide paying customers! 😀