evals were not broken when evals werent marketing goodhart, bad incentives
@swyx
-
Understanding FineWeb: Beyond CommonCrawl Dataset Architecture
By
–
yeah once this paper was released i was like “ohhh so fineweb isnt simply commoncrawl plus plus” and it all clicked into place. @eugeneyan pointed me to this apple paper we’ve talked about on the pod
-
Low-Code and Light Agentic Systems Will Win the Market
By
–
good riff, disagree on conclusion. i think its low code and light agenty systems like @juliusai and @wordware_ai and @cognition_labs that win
-
Tech Earnings Season: Market Bifurcating AI Winners and Losers
By
–
btw if you as a developer have not been paying attention to the tech earnings season, look out. basically market is bifurcating ai winners and losers hard rn. if you’re not an ai arms dealer or ai consumer company we basically just flipped to a bear market this week currently
-
RAG Retrieval Optimization Through Representation Engineering and Adapters
By
–
took a while to groq but great clear explanation here! an analogy could be that this is @voooooogel style rep eng applied to rag retrieval queries instead of inference outputs? adapters for everything. you could maybe even do adapters for multimodal data! as chameleon style
-
Long Context Window Progress: From 65k to 1M Tokens in One Year
By
–
quite incredible to see the goalposts of long context move in the past 1 year. May 2023: asking @jefrankle and @abhi_venigalla about their 65k+ "Llongboi" model May 2024: @markatgradient casually extending Llama 3 to >1m tokens with ~perfect NIAH and mainstream ai engineers
-
Questioning the legitimacy of Alibaba virtual try-on models
By
–
oh you do virtual try ons! just wondering – have you seen if any of the Alibaba models are legit? or are they vaporware?
-
Jeremy Howard’s Third Latent Space Podcast Appearance Coming Soon
By
–
we just recorded @jeremyphoward
's 3rd appearance on @latentspacepod (after the NeurIPS guest spot https://
x.com/latentspacepod
/status/1741160693582504275
…) and i have to say this one was EVEN MORE BANGING than the last. the world is not ready for what jeremy is about to drop in the next few months -
Curated AI Podcasts and Newsletters Resource List
By
–
periodic reminder that i do maintain a list of technically focused ai podcasts and newsletters here https://
github.com/swyxio/ai-note
s/blob/main/Resources/GoodAIPodcastsandNewsletters.md
… obviously its hard to cover math and images on voice format but the people who #learninpublic are heroes for helping the rest of us keep up -
GPT-3 Hyperparameters Analysis and Model Scaling Expectations
By
–
ah ok these are gpt3 hparams. it sounds like this alone would be enough to beat GPT2, and its unknown if 290B more tokens would get it to match or beat GPT-3 Small looking forward to the 1.5b – fascinating to see these all documented and taught live!!