Meta comes out looking so good from this. Meta’s AI division FAIR has consistently outmaneuvered Google in the open source deep learning community, starting with PyTorch and now with LLaMA.
AI
-
Custom Language Models Spawn New Generation of AI Startups
By
–
An entire generation of startups are going to be born and grow on the back of these technologies because they are so lightweight and cheap to make custom implementations. I have yet to talk to a company deploying language modeling that is not playing around with custom models.
-
Open Source LLaMA Models Achieving Foundation Model Performance
By
–
What is new is just how many times this has been applied to LLaMA and other models to squeeze near-equivalent foundation model performance out of a small number of parameters. But that’s basically what happens when the cat comes out of the bag in open source.
-
GPT Quantization and LoRA: Historical Context of Optimization Techniques
By
–
Those techniques are not explicitly new—GPT quantization goes back to October 2022 (and Google released a paper about leveraging sparsity in training in 2021). LoRA shows up in a paper in 2021.
-
LoRA and Quantization: Google Shifts from Big Numbers Approach
By
–
One of my big takeaways is just how important LoRA—basically a way to simplify and reduce the cost re-training models—and quantization for more compact models is here. Google has traditionally taken the Big Numbers approach to model development so it’s at minimum notable.
-
Geoff Hinton: The Old Dude Who Created AI
By
–
With all due respect, Geoff Hinton should change his Twitter bio to
— Gautam Kamath (@thegautamkamath) 5 mai 2023
"The old dude who created AI" – @SnoopDogg https://t.co/o4dffJNwRuWith all due respect, Geoff Hinton should change his Twitter bio to
"The old dude who created AI" – @SnoopDogg -
Company Age Comparison: Tech Giants from OpenAI to Nokia
By
–
Companies by age: OpenAI: 8 years Twitter: 17 years Facebook: 19 years Tesla: 20 years Google: 25 years Amazon: 29 years Apple: 47 years Microsoft: 48 years Sony: 77 years IBM: 112 years Nintendo: 134 years Nokia: 158 years v/
@spectatorindex -
AI’s Free Pass: Data Access Before Platform Restrictions
By
–
Now for the actual column, we're talking about how AI basically got a free pass all the way to launching an industry-breaking technology. The platforms where that data comes from are planning on gating it after the fact, but AI is following a time-honored tradition in tech.
-
Google AI Memo Leak: Quick Thoughts and Analysis
By
–
Today's issue of Supervised is a little more wonky as I want to address the leaked Google AI memo. I'm addressing it in a future issue but you can find some quick thoughts up top:
-
42-Hour Work Days: AI Automation and Labor Ethics
By
–
I mean.. realistically 42 hour work days is all you need!