…but it works we try a bunch of baselines like selecting data based on likelihood or even the top-k per-class our method (SELECT) can train a model to similar performance, with access only to model weights and some web data
CODE
-
Scaling Per-Example Gradients: Algorithm Efficiency Improvements
By
–
our algorithm is a bit complicated, mostly because computing per-example gradients is hard to do at scale so we make some efficiency improvements:
– computing grads w vmap
– only using last-layer grads (which are still big, in the case of LMs)
– projecting them to a smaller dim -
Finetuning AI Models: Gradient-Based Data Selection Strategy
By
–
to do this, you need TWO sets of model weights: the initial model and a finetune this is realistic. open-weights models often come with two checkpoints instead of one-shot generating data from weights, we select data from the web with gradients that point along the model diff
-
FastDiffusion course teaches comprehensive diffusion model knowledge
By
–
If you finish the fastdiffusion course you'll know a *lot* about diffusion 😀
-
English Replaces Code: Kids Build Apps with Natural Language
By
–
English is the new coding language. Kids are "vibe coding" apps by describing what they want. With things like @FactoryAI you can create entire apps with English words. How magical is that.
-
Software 3.0 Consuming Software Itself: The Next Evolution
By
–
@pmarca famously said 'software is eating the world.' Now, Software 3.0 is eating software itself. Software 1.0 and 2.0 are being absorbed into something more powerful. What happens next?
-
Codex Averages 10k Pull Requests Daily Over 35 Days
By
–
Codex has averaged 10k pull requests per day over the past 35 days:
-
Carbon Atoms as Patterns: Semantics Through Relational Change
By
–
A carbon atom is a pattern, its semantics are defined by how its presence changes other patterns.
-
Codex to Speed Up with Env Snapshots
By
–
WIP : Codex will be able to execute its tasks faster as it will be possible to save env snapshots. * Currently in development
-
Cursor Adds Slack Integration to Dashboard
By
–
To try it, head to http://
cursor.com/dashboard > Integrations > Slack.