Not sure about the context, but it probably refers the negative sample in a contrastive learning setting? The positive sample is similar to query, the negative sample is different from the query.
@rasbt
-
Scarcity of Multi-Turn Q&A Datasets in Machine Learning
By
–
I am actually not sure, that's a good point actually. In that sense, there also aren't that many multi-turn Q&A datasets out there (there are some, but maybe only 5% of all Q&A datasets are multi-turn)
-
Gratitude for Help with ML and AI Learning Journey
By
–
Thanks, and it's super nice to hear that it helped getting into ML & AI!!
-
Anticipating Alpaca Moment for Thought Tree Datasets in 2025
By
–
100%! Particularly, I am hoping for an Alpaca moment for "thought tree" datasets in 2025.
-
Chapter 3 Technical Complexity and Multi-Head Attention Implementation
By
–
It say Ch 3 might be the most technical one (like building the engine of a car) but it gets easier from here! You were wondering about some of the design choices. The implementation follows the original, popular, and widely used multi-head attention one (so we can also load
-
Improving LLM Reasoning: Test-Time Compute Scaling Over Parameters
By
–
And since o1 and the upcoming o3 are such important topics these days, I wanted to share a few good intros into improving the reasoning capabilities of LLMs I read earlier this year: 1. "Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters"
-
Open-Weight Reasoning Models Development Predictions for 2025
By
–
It will take time, but as both a wish and a prediction for 2025, I hope to see more interesting approaches to developing open-weight (o1 and o3-like) reasoning models through improved post-training recipes, data, and inference compute.
-
Tokenizer Implementation Limitations in Educational Materials
By
–
Thanks for sharing those, it's really nice seeing the chapters through the lens of a reader! Regarding the chapter 2 nit. It’s more of a shortcoming of the tokenizer implementation (it’s a very simple one for educational purposes). The reason why I copied it as is from the
-
DeepSeek-V3 achieves superior performance with 11x less compute than Llama
By
–
> E.g. Llama 3 405B used 30.8M GPU-hours, while DeepSeek-V3 looks to be a stronger model at only 2.8M GPU-hours (~11X less compute). Super interesting! And DeepSeek was trained in H800’s which are probably also a tad (or noticeably?) slower than Meta’s H100’s.
-
Voice assistants prove invaluable during medical emergency in Japan
By
–
Never liked voice assistants and only used Siri when I absolutely had to, like when driving. But last month when I ended up in an ER in Japan that was the only way to communicate with the doctors and will forever appreciate it (not ChatGPT but some voice translation app)