Wanna replace Anthropic/OpenAI? START WITH THIS The bible for running LLMs locally is now available online to read for free Covers what to use on – Laptop / edge / odd hardware
– Mac-first workflows
– Single RTX GPUs
– 2-4+ NVIDIA / CUDA GPUs
– General production serving
–
AI HARDWARE
-
Free online guide for running LLMs locally on all hardware
By
–
-
Every enterprise will have its own model-harness-sandbox-eval flywheel
By
–
Every enterprise will have its own model-harness-sandbox-eval flywheel with token value per watt optimization. This is the future. Simple reason: tacit knowledge about the domain and customers and their workflows that the company uniquely understands and has built trust around.
-
Skepticism about OSS owning high-end model hardware
By
–
Those of you who are putting all your hope in OSS – you really think that if they are serious about all of this they’ll let you own the hardware that can run the top end models?
-
Microsoft’s Singularity: 100,000 GPU Planet-Scale AI Infrastructure
By
–
Singularity: 100,000 GPUs. AI Infrastructure at Planet-Scale! Singularity is Microsoft’s distributed scheduling AI infrastructure, designed for the highly efficient and reliable execution of deep learning training and inference. Maximizing accelerator utilization without
-
Cerebras to share more Gemma 4 31B information updates
By
–
We will update more Gemma 4 31B information as we go! https://
inference-docs.cerebras.ai/models/gemma-4
-31b
… -
SambaNova thanks The Information, focuses on fastest AI inference
By
–
Thank you @theinformation
! We're focused on one thing: building the fastest, most efficient infrastructure for AI inference. As agentic AI scales, we believe premium inference will define the next era of computing. -
The defining metric of the 21st century: Intelligence per watt
By
–
The defining metric of the 21st century is Intelligence per watt. (IPW)
— Nina Schick (@NinaDSchick) 26 juin 2026
The throughput of Intelligence divided by the power consumed to produce it. pic.twitter.com/wcixNgdA3UThe defining metric of the 21st century is Intelligence per watt. (IPW) The throughput of Intelligence divided by the power consumed to produce it.
-
Offline voice assistant on tiny Axelera AI Mini PC
By
–
A full voice assistant, running with no internet connection at all, on a tiny, self-contained device you can put anywhere. This is the new Axelera AI Mini PC running Llama 3.2 1B as the language model, with separate speech-to-text and text-to-speech models alongside it, all on… pic.twitter.com/MUXkSdlJ7P
— Axelera AI (@AxeleraAI) 26 juin 2026A full voice assistant, running with no internet connection at all, on a tiny, self-contained device you can put anywhere. This is the new Axelera AI Mini PC running Llama 3.2 1B as the language model, with separate speech-to-text and text-to-speech models alongside it, all on
-
Local open-weight LLMs: 30B MoE sweet spot at 40 tok/sec
By
–
Have been taking different local open-weight LLMs for a test drive in different harnesses (Qwen-Code, Codex, Claude Code). 30B Mixture-of-Expert models are kind of a nice sweet spot and can solve challenging problems. And they get roughly 40 tok/sec on a Mac or DGX Spark, which
-
Heterogeneous disaggregated inference is the future of AI
By
–
Training builds models. Inference builds businesses. At @deeptechweek SF, our Chief Product & Strategy Officer Abhi Ingle shared why heterogeneous, disaggregated inference is the future of AI, and why "more intelligence per joule" is the metric that matters. @LipBuTan1