Meta comes out looking so good from this. Meta’s AI division FAIR has consistently outmaneuvered Google in the open source deep learning community, starting with PyTorch and now with LLaMA.
@mattlynley
-
Custom Language Models Spawn New Generation of AI Startups
By
–
An entire generation of startups are going to be born and grow on the back of these technologies because they are so lightweight and cheap to make custom implementations. I have yet to talk to a company deploying language modeling that is not playing around with custom models.
-
Open Source LLaMA Models Achieving Foundation Model Performance
By
–
What is new is just how many times this has been applied to LLaMA and other models to squeeze near-equivalent foundation model performance out of a small number of parameters. But that’s basically what happens when the cat comes out of the bag in open source.
-
GPT Quantization and LoRA: Historical Context of Optimization Techniques
By
–
Those techniques are not explicitly new—GPT quantization goes back to October 2022 (and Google released a paper about leveraging sparsity in training in 2021). LoRA shows up in a paper in 2021.
-
LoRA and Quantization: Google Shifts from Big Numbers Approach
By
–
One of my big takeaways is just how important LoRA—basically a way to simplify and reduce the cost re-training models—and quantization for more compact models is here. Google has traditionally taken the Big Numbers approach to model development so it’s at minimum notable.
-
AI’s Free Pass: Data Access Before Platform Restrictions
By
–
Now for the actual column, we're talking about how AI basically got a free pass all the way to launching an industry-breaking technology. The platforms where that data comes from are planning on gating it after the fact, but AI is following a time-honored tradition in tech.
-
Google AI Memo Leak: Quick Thoughts and Analysis
By
–
Today's issue of Supervised is a little more wonky as I want to address the leaked Google AI memo. I'm addressing it in a future issue but you can find some quick thoughts up top:
-
Pinecone targets double-digit millions ARR by end 2026
By
–
(To clarify, double-digit millions in ARR by the *end of the year* for Pinecone)
-
Data Stack Optimization Race for AI Models and Applications
By
–
So once again we have another divisive topic here as there's a new race to recreate the data stack in a way that's optimized for AI as large language and diffusion models enter every aspect of our lives—from enterprise tech to consumer applications.
-
Vector Databases in Modern Data Stack: Redis, SingleStore, PGvector
By
–
And of course this is once again in the feature-vs-product debate, like all previous re-inventions that plug into the modern data stack. Redis has a vector DB component, SingleStore is in the space, and there are open source tools like PGvector.