But that sometimes breeds innovation—like in the case of LLaMA. And scrappy developers without access to massive training clusters like Google or OpenAI have to find ways around the problem. That's led to the widespread adoption of techniques like quantization and LoRA.
@mattlynley
-
Startups Lower ML Model Training and Serving Barriers
By
–
As the cultures change here—at Meta or otherwise—the net beneficiary is likely to be the startups. Companies like MosaicML/Modal/Anyscale are trying to drop the barrier to training and serving ML models, and a lot of very cool research is being done on compact models.
-
FAIR’s Compute Challenges Compared to Google and Industry Peers
By
–
FAIR has sometimes not been seen as the destination research organization for AI, in part because historically developers don't get the level of compute that peers get at, say, Google. (Though I do hear even TPUs are more difficult to get a hold internally these days.)
-
AI Hype Reshapes Research Talent Competition at Big Tech
By
–
The hype around AI products is quickly changing the way research divisions work within larger orgs. Companies like Meta and Google have spent the last decade wooing some of the smartest minds in machine learning out of academia with the promise of working on cutting-edge tech.
-
ML Challenges Drive Development of Hyped Technologies
By
–
They join those organizations with the hopes of getting direct exposure to some of the most difficult problems in ML, like reinforcement learning. But most of those problems are now directly applicable to one of the most-hyped technologies in decades.
-
Meta Refocuses on Commercial Products, Cuts Bio Research
By
–
Case in point, the WSJ over the weekend published a piece on how Meta is refocusing its efforts to serve commercial products. My understanding is some of its more "out there" projects will be facing scrutiny. Its bio research team, for example, is one I've heard was impacted.
-
Has Anyone Beaten Textbooks Are All You Need Performance
By
–
Since it’s been 24 hours has someone already beaten the performance in textbooks are all you need yet?
-
Open Source Language Models Race Gains Expert Attention
By
–
Well there we go. I’d heard it was at least 4 experts but more mechanics on how it works out here. (Prev reported on it last week: https://
supervised.news/p/the-race-for
-open-source-language
…) -
Apple releases second major AI announcement in two weeks
By
–
so are we going to talk about how with byteformer that's two from apple in the past 2 ish weeks? https://t.co/4mrvkLFJSL
— Matthew Lynley (@mattlynley) 16 juin 2023so are we going to talk about how with byteformer that's two from apple in the past 2 ish weeks?
-
Major AI Stories: LeCun Interview, Nvidia Profile, Bard Safety, Section 230 Bill
By
–
Stories featured today from @radioproducer (Yann LeCun interview), @AustinCarr + @ianmking (profile of Nvidia), @JLDastin + @annatonger (Google warns against putting sensitive info in Bard), @ashleyrgold + @AndrewSolender (First look at bill denying section 230 for AI)