Btw it's not a DeepSpeed bug, more like something to address in the finetuning script! Sry for the confusion!
MACHINE LEARNING
-
BLEU Score Criticism Ignored Transformer Discovery
By
–
For decades people have been complaining about BLEU score not being a good metric for Machine Translation, and yet it is the metric that helped us identify the Transformer .
Haters gon' hate -
LMFlow-CHAT: Robin Series V2 Language Model Upgrade
By
–
LMFlow-CHAT demo: https://
huggingface.co/spaces/Optimal
Scale/Robin-7b
… fully upgraded Robin Series V2 language model carried out in-depth fine-tuning based on the entire LLaMA series, including 7b, 13b, 33b, 65b, all of which have achieved pleasing results. Robin-7b scored 51.7 in the OpenLLM standard -
Meta dominates AI landscape with PyTorch and LLaMA models
By
–
Meta has consistently outmaneuvered Google (and OpenAI/Microsoft by extension) in AI, both open source and the sheer volume of research it releases on a regular basis. It's pretty much locked up the ML dev ecosystem with PyTorch, and is well on its way w/ LLaMA.
-
Voicebox: Generative AI for High-Quality Audio Creation and Editing
By
–
Like other generative AI work, Voicebox is able to create high-quality outputs from scratch or modify samples, but instead of images/video, it produces high-quality audio. Unlike autoregressive models, it can modify any part of a given sample — not just the end of a clip.
-
Voicebox Outperforms Specialized AI Models with In-Context Learning
By
–
Voicebox generalizes across tasks and outperforms single-purpose AI models through in-context learning. It beats the previous state of the art in a number of ways. Better word error rates Improved audio similarity Up to 20x faster performance
-
Meta AI Introduces Voicebox Generative Speech System
By
–
Introducing Voicebox, a new breakthrough generative speech system based on Flow Matching, a new method proposed by Meta AI. It can synthesize speech across six languages, perform noise removal, edit content, transfer audio style & more. More details on this work & examples
-
DeepSpeed Stage 3 LoRA Bug and FSDP Migration Plans
By
–
Theoretically, you can get it to work if you use deepspeed stage 3 with offloading. But I see there is currently a little bug for stage 3 and LoRA. I think we watned to switch from DeepSpeed to FSDP anyway though. PS: Glad you like my book!
-
How Artificial Intelligence Could Fundamentally Change Certain Types of Work
By
–
How artificial intelligence could fundamentally change certain types of work
#AI #AIio #BigData #ML #NLU #Futureofwork @ahier @guzmand
@DavidBrin @denisegarth @dez_blanchfield @diioannid @DioFavatas @gerald_bader http://
ow.ly/S2nG30svHST -
Machine Learning as Configuration Art: A Critical Perspective
By
–
Sometime machine learning feels like the art of wiring together config options.