Fine-tune with Spectrum Dolphin models use this technique to target the most important layers to fine-tune, like in their recent Qwen2-72B version https://
huggingface.co/cognitivecompu
tations/dolphin-2.9.2-qwen2-72b
… You can use their repo to identify the layers and the "unfrozen_parameters" key in your Axolotl config.
LLMS
-
Fine-tuning LLMs with Spectrum: Targeting Key Layers Efficiently
By
–
-
Running LLMs on MacBook M1 Pro with LM Studio
By
–
Running on Macbook M1 pro (16GB RAM) using the @LMStudioAI.
— Shubham Saboo (@Saboo_Shubham_) 12 juin 2024
If you find this useful, RT to share it with your friends.
Don't forget to follow me @Saboo_Shubham_ for more such LLMs tips and tutorials.https://t.co/dxyCP5VjTmRunning on Macbook M1 pro (16GB RAM) using the @LMStudioAI
. If you find this useful, RT to share it with your friends. Don't forget to follow me @Saboo_Shubham_ for more such LLMs tips and tutorials. -
Llama-3 vs Qwen-2: Local LLM Comparison Guide
By
–
Compare Llama-3 and Qwen-2 running locally on your computer (100% free and without internet) pic.twitter.com/TOBPHNEiZm
— Shubham Saboo (@Saboo_Shubham_) 12 juin 2024Compare Llama-3 and Qwen-2 running locally on your computer (100% free and without internet)
-
Llama 3 Chat Template Optimization for Fine-tuning
By
–
Thanks actually I realised that Axolotl had a Llama 3 chat template after this finetune. It'd be better to use it instead of chatml for sure. It would be better but tbh I've milked this recipe a bit too much 🙂
-
Phi-3 MoE Implementation Discussion
By
–
I haven't checked but that sounds good indeed, you might be able to directly do a phi-3 MoE with this
-
Elon Musk sues OpenAI over Microsoft alliance direction
By
–
El salseo del día: Elon Musk, quien en su momento colaboró en fundar OpenAI y la abandonó en 2018 por discrepancias sobre la dirección que estaban tomando en su alianza con Microsoft -camino que les ha llevado a los LLMs-, ahora estaría interponiendo una demanda contra OpenAI.
-
Leaks and Rumors Surrounding GPT-5 Capabilities
By
–
More examples:
— 🚨 AI News | TestingCatalog (@testingcatalog) 11 juin 2024
"A random X account posting AI leaks about GPT-5" pic.twitter.com/r90PxQ9IHzMore examples: "A random X account posting AI leaks about GPT-5"
-
Understanding Hidden Custom Instructions in AI Models
By
–
Custom instructions are there but hidden 😛
-
User evaluation of Claude Opus via Perplexity UI
By
–
I don't see any. I use Opus via Perplexity instead. The UI feature set is super basic, unfortunately.
-
Open-Source Agentic Machine Translation Demonstration Released
By
–
I think AI agentic machine translation has huge potential for improving over traditional neural machine translation, and am releasing as open-source a demonstration I'd been playing with as a fun weekend project. Using an agentic workflow, this demonstration (i) Prompts an LLM