Meta's LLaMA model has turbo-charged the open source developer community, even if it's still restricted to research use. I'm hearing that a full OSS version will be dropping soon and that Meta is exploring models with fewer parameters than its smallest 7B model.
@mattlynley
-
Vectors All the Way Down: Understanding AI Representations
By
–
Also this tweet is a vector and if you can see it you too are a vector. It's vectors all the way down.
-
Meta ImageBind Open Source Multimodal AI Model Research
By
–
Everything is a vector! https://
theverge.com/2023/5/9/23716
558/meta-imagebind-open-source-multisensory-modal-ai-model-research
… -
Low-Parameter AI Models Reshaping the Industry Landscape
By
–
Supervised coming out a bit earlier today—we're talking about low-parameter models and the impact they're going to have in AI. Particularly with Google I/O (and as a bonus, WWDC) around the corner. https://
supervised.news/p/a-model-in-y
our-pocket
… -
Google Open-Source Panic and the AI Data Race
By
–
First week of Supervised is a wrap! Thank you to everyone for all your support—super excited to keep this rolling and hope you all enjoy the stories coming next week! In the mean time, here’s what’s from this week: https://
supervised.news/p/googles-open
-source-panic-and-ais
… https://
supervised.news/p/the-mad-dash
-for-the-new-ai-data
… -
Early OSS lightweight models testing formats private beta
By
–
As a funny aside, one of the early private beta stories I circulated among friends a few months ago as I was testing formats was literally titled “a model in your pocket” and about low-param OSS models. Whoops.
-
JAX Omission Raises Questions About AI Training Technology Strategy
By
–
There’s no mention of JAX. That was a little weird as they own an extremely powerful next-generation training technology in the same way they owned TensorFlow at first.
-
Google’s Strategic Missteps in AI and Open Source Innovation
By
–
Google keeps falling asleep at the wheel here. It didn’t productize MapReduce, allowing Hadoop to happen. It let PyTorch blossom and take over the DS/DL community. And now it has basically let competitors run off with the Transformers technology.
-
GPT-4 Architecture: Parameter Count and Competitive Moat Analysis
By
–
OpenAI is mentioned in here as having no moat, but OpenAI was also intentionally silent about the parameter count of GPT-4. I know some people have been flinging around a 1T parameter number for GPT-4 but it’s very possible that it isn’t a single 1T model and instead 1T total.
-
Meta’s LLaMA Leak: Legal Implications and Open Source Adoption
By
–
And you have to appreciate the stance here which is basically like “what is Meta going to do, sue all the people using the leaked LLaMA model?”