FYI for anyone trying to get the most out of Mixtral 8x22B: don't try to prompt it like you prompt other models. It's a base model, not a tuned model. It isn't trained to follow instructions. Instead, provide examples of the behavior you're looking for. Far better results.
@mattshumer_
-
Build Any System: Community Ideas Challenge
By
–
Someone give me an idea for a system they want me to build, and I’ll record myself building it
-
Using AI to Write Code for AI Systems
By
–
Would anyone want me to post a video showing how I use AI to write code for AI systems? Poll isn’t working for some reason so let me know in the replies.
-
Custom Mixtral 8x22B Integrated into HyperWrite Platform
By
–
I've hacked a custom Mixtral 8x22B into the @HyperWriteAI platform. With access to real-time info. Will spend some time iterating and improving it, then we'll roll it out to our user base!
-
Mixtral 8x22B Base Model Shows Strong Assistant Capabilities
By
–
From testing the Mixtral 8x22B base model, it seems pretty clear that a lot of instruction/chat data was trained into the base model. Even without fine-tuning, it's already capable of acting as a pretty great assistant, just via prompting.
-
User Requests Additional A100 GPUs for Bixtral Model Testing
By
–
Hey @ClementDelangue would you mind adding two extra A100s (total 4) to my account? Want to run some tests on Bixtral (big Mixtral) using HF Endpoints.
-
Using AI Music Tools for Creative Song Inspiration
By
–
As someone who loves making music (even though I'm not very good at it), one of the best use-cases I found for @udiomusic is using it as inspiration for writing new songs. Infinite ideas, at the push of a button.
-
Mixtral-8x22B Fine-tuning Configuration with LoRA Parameters
By
–
base_model: mistral-community/Mixtral-8x22B-v0.1
model_type: AutoModelForCausalLM
tokenizer_type: LlamaTokenizer
trust_remote_code: true load_in_8bit: false
load_in_4bit: false
strict: false unfrozen_parameters: – ^lm_head.weight$ – ^model.embed_tokens.weight$ – -
Selective Layer Freezing for Language Model Fine-tuning Strategy
By
–
Freezing all layers except these: – ^lm_head.weight$ – ^model.embed_tokens.weight$ – model.layers.4[4-9]+.block_sparse_moe.gate – model.layers.4[4-9]+.block_sparse_moe.experts – model.layers.5[0-5]+.block_sparse_moe.gate – model.layers.5[0-5]+.block_sparse_moe.experts