the models are homogeneous, the hardware SKUs are homogeneous, so the overall perf tuning/search space is relatively small. So, the bounded search makes it less moaty — competitors are also searching within the same bounded space.
I cover more on this in my upcoming Latent Space
@soumithchintala
-
Bounded Search Space Limits Competitive Moat in AI Hardware
By
–
-
LLM Inference Margins Race to Bottom No Sustainable Moat
By
–
Great LLM Inference benchmark — all the right variables are in there.
— Soumith Chintala (@soumithchintala) 26 janvier 2024
Now, the race to the bottom begins, because in LLM Inference there isn't a sustainable long-term technology or software moat. IMHO the margins are going to resemble grocery stores and laundromats…
Some of… https://t.co/pR5xVIpik3Great LLM Inference benchmark — all the right variables are in there.
Now, the race to the bottom begins, because in LLM Inference there isn't a sustainable long-term technology or software moat. IMHO the margins are going to resemble grocery stores and laundromats…
Some of -
Google’s Distribution Effects Creating Runaway AI Advantage
By
–
Very interesting update.
Maybe Google's distribution effects are starting to drive a runaway advantage… -
Chris Olah’s Accessible and Exciting AI Posts and Papers
By
–
chris olah's posts and papers are highly accessible and exciting. maybe thats a cause.
-
H100 GPU Capacity Math: Actual Deployment Numbers 2024
By
–
the 350k H100s by the end of 2024 includes the H100s that we already have. Now do the math
-
Meta deploys 600k H100-equivalent GPUs by end of 2026
By
–
Can finally talk some GPU numbers publicly By the end of the year, Meta will have 600k H100-equivalent GPUs.
Feel free to guess what's already deployed and being used ! -
GPU Kernel Optimization Arbitrage Among AI Infrastructure Providers
By
–
There's very cool arbitrage happening right now — with @hippoml_com @FireworksAI_HQ @togethercompute — where they're writing GPU kernels to improve efficiency on configurations of hardware + workloads that are important but not looked at by large providers.
This is obviously a -
GPT-4 Remains Superior to All Other AI Models
By
–
not that I know of. GPT-4 has been far ahead of anything else (closed or open) in my experience, practically.
-
GPT-4 as PyTorch Development Co-pilot: Solving Conv2d Tiling
By
–
over the last couple of days, I've been answering PyTorch forum questions using GPT-4 as a co-pilot. The coolest/hardest one was to write a tiled Conv2d (accounting for edge effects) — i kept telling GPT how it was wrong, and it basically gave me a working solution.
-
Runway ML Magazine Bridges Art and AI Communities
By
–
came back home to this beautiful first issue from @runwayml that bridges the Art and AI communities. A quote from the mag:
"if the result blows you away, it's because traditional steps, planning and human talent are still behind it." – Ricardo Vilavicencio, the creator of the