There are two kinds of people in AI: those who are happily surprised a model would know random facts like these (worth burning compute and parameters), and those who think it's a complete waste of both for a dense core.
@alexjc
-
Quality over quantity: improving training datasets for better AI models
By
–
Great idea for a metric to further improve what datasets the models train on. It likely leads to an answer that is not web-scale crawling… Less data is often better, better data takes less.
-
Meta layoffs signal shifting competitive landscape in AI research
By
–
Isn't this a sign of the market becoming more competitive, the golden age of open research from U.S. corporations being increasingly over, and now researchers who would be publishing anyway (without it costing Meta) are being let go?
-
Meta’s Soft Tokens Enable LLMs to Invent Recursive Representations
By
–
This paper from Meta about "Soft Tokens" in RL is interesting; it allows LLMs to invent their own non-discrete (recursive) representations in order to solve problems better… Results are mixed though: it's only a few percent better on GSM8k from pass@4 onwards, and pass@32 just
-
AI Browser Strategy Shifts Scraping Liability to Users
By
–
The reason AI companies are rushing to release browsers: they don't want the responsibility / liability of scraping on their servers. They need to push that to the users! We'll be moving into an ever more gated internet soon…
-
Fair Use Defense Burden of Proof in AI Legal Cases
By
–
I'm so glad for this, looking forward to reading! Courts may drag their feet, but specifically when relying on a Fair Use defense, the burden of proof for the absence of market harms falls on the defendants.
-
Getting Coding Agents to Produce Shippable Quality Code
By
–
Is this the only way to get coding agents to produce shippable quality code?
-
Scaling AI Without Purpose: A Critique of Growth
By
–
Scaling just for the sake of it even if it produces nothing useful? Heh.
-
Debating AI Topics Without Clear Definition Consensus
By
–
It's amazing how the topics can be debated without there being much of a consensus on what's being debated…
-
The Bitter Lesson: Compute versus Other Approaches in AI
By
–
Without checking, what is the message behind the "Bitter Lesson", in your opinion? (a) all other things being equal, using more compute is better.
(b) more compute is better than all the other things put together.