AI Dynamics

Global AI News Aggregator

About

Layer Replication and Pruning Techniques for Model Optimization

These experiments re-confirm a lot of recipes discovered by doing frankenmerges. – Replicating layers is helpful for creative tasks but not for reasoning (in general) – Changing the layer order doesn't work – Pruning works best with middle layers but still requires healing

→ View original post on X — @maximelabonne