a related point is that every time an AI lab finishes training a model, the US government probably downloads it over the wire immediately the idea that a five-year old AI lab has fully secured itself from nation-state hacking is a bit naive https://
x.com/willdepue/stat
/willdepue/status/2068118253633737077
…
@jxmnop
-
US government likely downloads AI models right after training
By
–
-
Google’s privacy restrictions hinder Gemini’s performance versus rivals
By
–
There is a simple reason why Gemini is so much worse than GPT or Claude engineers at OpenAI or Ant can read incoming user queries. all the data is visible but at Google there are tons of privacy restrictions preventing ppl from looking at data basically building a model blind
-
Traditional ML background irrelevant for modern deep learning systems
By
–
endlessly fascinating how a traditional machine learning background is basically not that helpful for modern AI. we use deep NNs and do SGD with one of two losses. most day-to-day work lies in abstractions *on top* of this layer. everything is really just a massive system of
-
Fable and Mythos for data work, not huge dataset loops
By
–
to clarify, looping Fable over huge datasets wouldn't be a good use of compute for many reasons but much of what goes into better models is DATA WORK: eval design, rubrics, error analysis, repairing noisy data Fable can do this!
and Mythos will do this too -
Data work key to model improvement: evals, rubrics, error analysis
By
–
obviously, but u are missing the point i'm saying most of what makes models better is DATA WORK: evals, rubrics, error analysis, and so on Fable can do this, Mythos will do this, future models will do this
-
Huge leverage in improving data and xAI’s plan to fix pretraining typos
By
–
An underrated part of this discussion is that (a) there's huge leverage in improving data, and (b) there's no way Anthropic could safeguard this xAI could instruct Fable to look through EVERY row of pretraining data and fix any typos and errors. this probably the single
-
Debate on steering vector or LoRA in ML research
By
–
i'm hearing some mixed interpretations of this, is it Option A:
There is a steering vector or LoRA that fires only when you're working on cutting-edge ML research and makes the model dumber OR Option B:
If you try full finetuning, Claude will subtly convince you to use LoRA? -
Joke-telling AI progress plateaued since 2022 PaLM
By
–
the new Fable still can't tell a joke i think jokery evals plateaued with Google's PaLM models in 2022, no one has pushed SOTA since then maybe another 10 trillion parameters will do the trick!
-
Cursor’s frontier model with 100x fewer resources than Google
By
–
it is wild that Cursor trained a model closer to the frontier than Google with 100x fewer people and (guessing) ~100x less compute i am surprised this was even possible. also praying for the Gemini comeback ofc
-
AI crisis due to distance from datacenters and GPU whir
By
–
The AI world is in crisis precisely because man has strayed too far from his datacenters. one shouldn’t be able to start a 1000-gpu training run without hearing the GPUs start to whir. heretics