What is new is just how many times this has been applied to LLaMA and other models to squeeze near-equivalent foundation model performance out of a small number of parameters. But that’s basically what happens when the cat comes out of the bag in open source.
MACHINE LEARNING
-
GPT Quantization and LoRA: Historical Context of Optimization Techniques
By
–
Those techniques are not explicitly new—GPT quantization goes back to October 2022 (and Google released a paper about leveraging sparsity in training in 2021). LoRA shows up in a paper in 2021.
-
LoRA and Quantization: Google Shifts from Big Numbers Approach
By
–
One of my big takeaways is just how important LoRA—basically a way to simplify and reduce the cost re-training models—and quantization for more compact models is here. Google has traditionally taken the Big Numbers approach to model development so it’s at minimum notable.
-
Geoff Hinton: The Old Dude Who Created AI
By
–
With all due respect, Geoff Hinton should change his Twitter bio to
— Gautam Kamath (@thegautamkamath) 5 mai 2023
"The old dude who created AI" – @SnoopDogg https://t.co/o4dffJNwRuWith all due respect, Geoff Hinton should change his Twitter bio to
"The old dude who created AI" – @SnoopDogg -
Andrew Ling to Discuss LLM Architecture Choices at FCCM Panel
By
–
Tune in on May 10 to hear Andrew Ling, PhD, Sr. Director, Software Engineering & ML Compiler participate in a FCCM panel about architecture choice for training and inference markets for Large Language Models (LLMs). Learn more at https://
fccm.org/panel-2023/ -
LearnCrunch Upcoming Courses: Tech Skills by Industry Experts
By
–
Upcoming courses at @LearnCrunch_ (including mine) help you build business-critical tech skills in cohort-based live courses by leading industry experts: http://
learncrunch.com
———
#MachineLearning #AWS #AI #DataScience #DataScientists #DataEngineers #DataViz #Python #Coding -
Free PDF Guide to Deep Learning Fundamentals Available Now
By
–
Download Free PDF! The Little Book of #DeepLearning: http://
fleuret.org/public/lbdl.pdf
————
#AI #MachineLearning #NeuralNetworks #GenerativeAI #BigData #DataScience #DataScientists -
Free Stanford CS246 Course: Mining Massive Datasets PDF and Slides
By
–
Download free PDF book (updated 3rd edition) *and* the lecture slides from the Stanford #ComputerScience course CS246: “Mining Massive Datasets” at https://
online.stanford.edu/courses/soe-yc
s0007-mining-massive-data-sets
…
——————
#BigData #DataScience #MachineLearning #AI #DataMining #DeepLearning #Algorithms #Mathematics -
Free Comprehensive Neural Network Design Textbook PDF Download
By
–
Classic but still gold >> Download free PDF of comprehensive 1000-page textbook >> Neural Network Design [2nd edition]: http://
hagan.okstate.edu/NNDesign.pdf
————
#BigData #DataScience #AI #MachineLearning #DeepLearning #NeuralNetworks #Mathematics #Algorithms -
Multimodal Deep Learning: Comprehensive 272-Page eBook Guide
By
–
“Multimodal #DeepLearning” http://
arxiv.org/abs/2301.04856
[Download 272-page PDF eBook]
———
#BigData #DataScience #AI #ML #MachineLearning #NLProc #ComputerVision #NeuralNetworks