Happy New Year! To kick off the year, I've finally been able to format and upload the draft of my AI Research Highlights of 2024 article. It covers a variety of topics, from mixture-of-experts models to new LLM scaling laws for precision:
@rasbt
-
Paperback Book Sale Announcement on Amazon
By
–
Haha, thanks for the shout out!! Btw the paperback is currently on sale on Amazon in case that helps (
https://
amazon.com/Build-Large-La
nguage-Model-Scratch/dp/1633437167/ref=tmm_pap_swatch_0
…) -
V100 GPUs Originally Designed for Molecular Dynamics Simulations
By
–
I am old enough to remember that V100 were actually designed with MD simulations in mind (or maybe it was their predecessor?) due to their fp64 support
-
Use Cases for Higher Precision Computing: Debugging, Education, Simulations
By
–
Debugging, educational use cases, maybe simulations you run in higher precisions (like MD and physics simulations)
-
Positive and Negative Samples in Contrastive Learning
By
–
Not sure about the context, but it probably refers the negative sample in a contrastive learning setting? The positive sample is similar to query, the negative sample is different from the query.
-
Scarcity of Multi-Turn Q&A Datasets in Machine Learning
By
–
I am actually not sure, that's a good point actually. In that sense, there also aren't that many multi-turn Q&A datasets out there (there are some, but maybe only 5% of all Q&A datasets are multi-turn)
-
Gratitude for Help with ML and AI Learning Journey
By
–
Thanks, and it's super nice to hear that it helped getting into ML & AI!!
-
Anticipating Alpaca Moment for Thought Tree Datasets in 2025
By
–
100%! Particularly, I am hoping for an Alpaca moment for "thought tree" datasets in 2025.
-
Chapter 3 Technical Complexity and Multi-Head Attention Implementation
By
–
It say Ch 3 might be the most technical one (like building the engine of a car) but it gets easier from here! You were wondering about some of the design choices. The implementation follows the original, popular, and widely used multi-head attention one (so we can also load
-
Improving LLM Reasoning: Test-Time Compute Scaling Over Parameters
By
–
And since o1 and the upcoming o3 are such important topics these days, I wanted to share a few good intros into improving the reasoning capabilities of LLMs I read earlier this year: 1. "Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters"