I trained a 4bit on a Llama base with alpaca dataset today. Reasoning was worse compared to traditional alpaca.
4-bit Llama Fine-tuning with Alpaca Dataset Performance Analysis
By
–
By
–
I trained a 4bit on a Llama base with alpaca dataset today. Reasoning was worse compared to traditional alpaca.