You can imagine a company coming to Facebook, saying it wants a custom version of LLaMA, and Facebook deciding to do the training/fine-tuning/serving for it. All the tech is there, along with the talent, DNA, and resources.
@mattlynley
-
Meta’s Untapped AI Business Opportunity Finally Emerging
By
–
It's built all this internally, but Facebook (sorry, Meta), has been on the hunt for a new business for years. It just happened to be sitting on some of the most popular AI tools in existence and one of the best AI research facilities in the world—and doing nothing with it.
-
Facebook builds ML-focused Google Cloud competitor with LLaMA
By
–
But there's another part of the story here: Facebook has built what seems to be a very early machine learning-focused prototype of.. Google Cloud. It has a training cluster technology it's developed, hardware for inference, and owns a massively popular OSS-ish LLM, LLaMA.
-
AI Chip Startups Face Setbacks Amid Market Challenges
By
–
Some of the hottest chip startups are now hitting snags. Graphcore laid off 160 people last year as a deal to supply Microsoft fell through, and SambaNova also went through some layoffs earlier this year. Cerebras has pushed hard into LLMs with its own OSS LLM models.
-
Big Tech Invests in GPU Superclusters for AI Training
By
–
Google allocated a portion of its i/o announcements to a new supercomputer—powered by H100s. Facebook's RSC announcement pointed out that it was powered by Nvidia A100s. Amazon and Microsoft have A100s. And while all have some form of custom silicon, it's largely in inference.
-
Nvidia’s Dominance: Why Startups Failed to Compete
By
–
With Facebook's announcement yesterday, the biggest players in the AI industry have coalesced around Nvidia for training. A pack of startups once sought to beat Nvidia, but the dream of beating Nvidia is a much taller one than imagined in 2016-2017.
-
Who Is Doing Most Interesting Self-Supervised Learning Work
By
–
Who is doing the most interesting work in self-supervised learning rn?
-
AGI Futures Presented More Eloquently Than Most
By
–
AGI futures presented more eloquently than most I've seen. https://
roonscape.ai/p/agi-futures -
OpenAI’s embedding model ada-002 faces Hong Kong University competition
By
–
It's also worth mentioning that OpenAI has a lot to lose here. Its foundation models are the standard right now (like 90% of the people I talk to use GPT-4). But its text embedding model, ada-002, isn't actually the best one available. That one comes out of Hong Kong University.
-
OpenAI OSS Tool Release Ahead of AI Regulatory Hearing
By
–
If you just look at a snapshot of the timeline of AI development, the releases get denser and denser as we arrive to the regulatory hearing on the 16th. OpenAI is potentially readying an OSS tool, but like Google's it might already be behind the trends in OSS.