also, embeddings *are* underrated
@jxmnop
-
Embeddings Are Underrated: Technical Deep Dive
By
–
# Embeddings are underrated (2024) just a really excellent piece of technical writing.
-
Aspiring to Master Technical Writing in Machine Learning
By
–
link here – http://
technicalwriting.dev/ml/embeddings/
overview.html
…
i aspire to write like this person someday… -
LLMs solve verifiable math and coding problems only
By
–
right now they do math problems that are verifiable, or coding problems where you can run the code and see if it worked and was correct it's very strange isn't it?
-
RL Verification Rewards vs SFT Synthetic Data Training
By
–
this isn't right. RL with verification-based-rewards is a different learning signal than generating synthetic data via language modeling and then using SFT, which is what you're describing
-
From Supervised to Reinforcement Learning: The Next AI Frontier
By
–
the AI labs spent a few years quietly scaling up supervised learning, where the best-case outcome was obvious: an excellent simulator of human text now they are scaling up reinforcement learning, which is something fundamentally different. and no one knows what happens next
-
Reasoning Tokens for Search and Information Retrieval
By
–
Reasoning Tokens Not for Needles, but for Haystacks
-
Needle in Haystack Papers: Which Research Study is Fake?
By
–
pop quiz! which one of these is *not* a real paper? (answer below) – Multimodal Needle in a Haystack
– Multilingual Needle in a Haystack
– Needle In A Multimodal Haystack
– Reasoning on Multiple Needles In A Haystack
– In Search of Needles in a Haystack
– Multiple Needles -
GPU Performance Issues With Sparsity in Machine Learning
By
–
this is nice but people have abandoned sparsity because it doesn't give you good performance in practice, since it doesn't run well on GPU
-
Stripe’s AI Model: Excellence and Statistical Literacy Lessons
By
–
it seems simultaneously true that (A) stripe built a very good model using cutting-edge AI and (B) this is a teaching moment about basic statistical literacy