AI Dynamics

Global AI News Aggregator

About

Mixtral-8x22B Fine-tuning Configuration with LoRA Parameters

base_model: mistral-community/Mixtral-8x22B-v0.1
model_type: AutoModelForCausalLM
tokenizer_type: LlamaTokenizer
trust_remote_code: true load_in_8bit: false
load_in_4bit: false
strict: false unfrozen_parameters: – ^lm_head.weight$ – ^model.embed_tokens.weight$ –

→ View original post on X — @mattshumer_