Thoughtful skepticism doesn't mean mindlessly challenging authority. It means carefully following the evidence.
RESEARCH
-

Mathematics Role in Machine Learning and Deep Learning Research
By
–
Shape, Symmetries, and Structure: The Changing Role of Mathematics in Machine Learning Research https://
bit.ly/3Z32DVn #AI #MachineLearning #DeepLearning #LLMs #DataScience -
Solving ARC-AGI Prize Daily Puzzle Challenge
By
–
The ARC-AGI prize is fun to do and somewhat relaxing, even though I don’t generally play casual games… @fchollet @arcprize ARC Prize Daily Puzzle
Task: 692cd3b6 6:52 sec 1 attempt Can you solve it? http://
arcprize.org/play -
DeepSeek-V3 achieves superior performance with 11x less compute than Llama
By
–
> E.g. Llama 3 405B used 30.8M GPU-hours, while DeepSeek-V3 looks to be a stronger model at only 2.8M GPU-hours (~11X less compute). Super interesting! And DeepSeek was trained in H800’s which are probably also a tad (or noticeably?) slower than Meta’s H100’s.
-
Human Information Processing: Receiving vs Transmitting Capacity Limits
By
–
Thanks Felix … I do think the "The Atomic Human" deals with this … I guess the point is that while we can receive a lot of information … we're constrained in how much we transmit — by their calculations … about 10 bits per second by mine about 30 …
-
RoPE and GQA Implementation in GPT to Llama Conversion Guide
By
–
Glad to hear that you've been enjoying it! Regarding RoPE and GQA, I've added some bonus material here a few months back where I implement it from scratch in the GPT -> Llama conversion guide: https://
github.com/rasbt/LLMs-fro
m-scratch/tree/main/ch05/07_gpt_to_llama
…
I was planning to add more, but unfortunately I am currently -
Dense vs Mixture of Experts: Technical Insights and Public Sources
By
–
Very interesting, are there any more intuition on dense vs MoE? Also hate to be that guy.. but is there a public source for this (to test okay if not)
-
Why models don’t know their own release version
By
–
A model can’t know its own release version because it’s usually only decided that a trained model *should* be a release version once it’s fully post-trained, red-teamed, etc. Human demonstrations are necessarily older. Claude I assume would learn it in RLAIF.
-
Elon Musk and xAI Team Show Strong Promise in AI
By
–
have quite a lot of faith in Elon plus xAI folks – they’re cracked AF!
-
Comment on system prompt and post-training practicality
By
–
Yeah sorry, was unclear this is coming from the system prompt — the reason I ask this is because it’s impractical to teach it in post-training. Added a comment to the thread:
