so xAI just 10x’d the amount of compute we use on RL and the models only got a tiny bit better are we just doing RL wrong? or is pretraining just inherently much more useful
MACHINE LEARNING
-
Distillation as Garbage Collection for Deep Neural Networks
By
–
Distillation is garbage collection for deep networks.
-
Fine-tuning versus System Prompts in AI Models
By
–
AFAICT this appears to be a result of fine tuning. I can't find any sign that this is part of the system prompt. It's also possible it's injected into the chat itself but then removed from the history.
-
GitHub Copilot Study: AI Transforms Developer Work and Earnings
By
–
This large study of 187k developers using GitHub Copilot finds AI transforms nature of coding. Coders focus: more coding & less management. They need to coordinate less, working with fewer people They experiment more with new languages, which would increase earnings $1,683/year
-
Cut Through AI Hype and Extract Real Value
By
–
How To Cut Through The Hype And Get Value From AI
#AI #AIio #AIInnovation #ML #DataScience #Futureofwork @timnitgebru @oriolvinyalsml @ceobillionaire @soumithchintala @waitin4agi_ @sallyeaves @bernardmarr http://
ow.ly/rcMw30sNAOU -
Training LLMs on AI Paper Tweet Summaries
By
–
train LLM on all AK tweet summaries of papers @_akhaliq @arankomatsuzaki and other AKs
-
AI Breakthroughs: Five Years of Scaling Evolution
By
–
AI breakthroughs in the last 5 years are all defined by finding the right axis to scale up 2021: scaling model parameters 2022: scaling data / tokens 2024: scaling inference time compute / CoT
2025: scaling ai researcher comp -
Better AI in 2025 Powered by Real-Time Data
By
–
Don't Just Expect More AI In 2025—Expect Better AI, Fueled By Real-Time Data
#AI #AIio #AIInnovation #ML #DataScience #Futureofwork @demishassabis @Ronald_vanLoon @TamaraMcCleary @geoffreyhinton @goodfellow_ian @jeffdean @erikbryn http://
ow.ly/Wujw30sNAOT -
Merged Experts in MoE: Coordination Without Joint Training
By
–
all experts are merged into an MoE. The anchor model ensures that all experts learn to coordinate, even though they are never trained on the joint dataset. this is rocket science wtf
-
Fourth European Workshop on Algorithmic Fairness Proceedings Published
By
–
Volume 294 Proceedings of the Fourth European Workshop on Algorithmic Fairness https://
proceedings.mlr.press/v294/ Is now available on PMLR