Fine-tune with Spectrum Dolphin models use this technique to target the most important layers to fine-tune, like in their recent Qwen2-72B version https://
huggingface.co/cognitivecompu
tations/dolphin-2.9.2-qwen2-72b
… You can use their repo to identify the layers and the "unfrozen_parameters" key in your Axolotl config.

→ View original post on X — @maximelabonne,