From testing the Mixtral 8x22B base model, it seems pretty clear that a lot of instruction/chat data was trained into the base model. Even without fine-tuning, it's already capable of acting as a pretty great assistant, just via prompting.
@mattshumer_
-
User Requests Additional A100 GPUs for Bixtral Model Testing
By
–
Hey @ClementDelangue would you mind adding two extra A100s (total 4) to my account? Want to run some tests on Bixtral (big Mixtral) using HF Endpoints.
-
Using AI Music Tools for Creative Song Inspiration
By
–
As someone who loves making music (even though I'm not very good at it), one of the best use-cases I found for @udiomusic is using it as inspiration for writing new songs. Infinite ideas, at the push of a button.
-
Mixtral-8x22B Fine-tuning Configuration with LoRA Parameters
By
–
base_model: mistral-community/Mixtral-8x22B-v0.1
model_type: AutoModelForCausalLM
tokenizer_type: LlamaTokenizer
trust_remote_code: true load_in_8bit: false
load_in_4bit: false
strict: false unfrozen_parameters: – ^lm_head.weight$ – ^model.embed_tokens.weight$ – -
Selective Layer Freezing for Language Model Fine-tuning Strategy
By
–
Freezing all layers except these: – ^lm_head.weight$ – ^model.embed_tokens.weight$ – model.layers.4[4-9]+.block_sparse_moe.gate – model.layers.4[4-9]+.block_sparse_moe.experts – model.layers.5[0-5]+.block_sparse_moe.gate – model.layers.5[0-5]+.block_sparse_moe.experts
-
Udio AI Music Generation Quality and Musical Output Comparison
By
–
In terms of stability, I agree, but the music from Udio just sounds more… musical…
-
AI Music Generation Quality Approaches Spotify Indistinguishability
By
–
I've been playing with @udiomusic for a few days now.
— Matt Shumer (@mattshumer_) 10 avril 2024
The quality of the music it generates is insane.
Stop whatever you're doing and listen to this.
It feels like we're less than a year away from AI music that is indistinguishable from what you hear on Spotify. https://t.co/xgdRKG2R8X pic.twitter.com/tn6gxIKt6HI've been playing with @udiomusic for a few days now. The quality of the music it generates is insane. Stop whatever you're doing and listen to this. It feels like we're less than a year away from AI music that is indistinguishable from what you hear on Spotify.
