High workload, running 120b models locally.
HARDWARE
-
16GB VRAM Doable for Standard Hardware Except Long Context Tasks
By
–
Maybe for very long context agent tasks, but otherwise, 16GB VRAM is pretty doable on standard hardware
-
Gemma 4 12B released, runs comfortably on 16GB VRAM for LFM newcomers
By
–
They should all run comfortably (if you have ~16 GB VRAM). I am quite new to LFM's, and Gemma 4 12B just came out today, so time will tell…
-
Four additions to open-weight local LLM on consumer hardware
By
–
It's been a while! 4 nice additions to the open-weight local-LLM-on-consumer-hardware ecosystem:
-
NVIDIA Local AI Agents Level Up with OpenShell and RTX Updates
By
–
Local AI Agents are leveling up across DGX Spark & RTX PCs. NVIDIA OpenShell is coming to Windows alongside new agentic AI optimizations and creator app updates—including NVIDIA Broadcast 2.2, plus upcoming RTX acceleration for Adobe apps and Blender. Learn More:
-
SambaNova post about Intel’s rack-scale agentic AI design
By
–
Agentic AI needs CPUs, GPUs, and AI accelerators working together. @TheRegister highlights @intel
's new rack-scale agentic AI designs and the first customer deployment of the Intel + SambaNova disaggregated inference blueprint through VC2. The result: GPUs handle prefill, -
Flash Attention: Hardware-Level SRAM Caching Achieves 7.6x Speedup
By
–
Flash attention involves hardware-level optimizations wherein it utilizes SRAM to cache the intermediate results. This way, it reduces redundant movements, offering a speed up of up to 7.6x over standard attention methods. Check this
-
Flash Attention: Efficient Global Attention via GPU Memory Optimization
By
–
2) Flash Attention This is a fast and memory-efficient method that retains the exactness of traditional attention mechanisms, i.e., it uses global attention but efficiently. The whole idea revolves around optimizing the data movement within GPU memory. Let's understand!
-
SambaNova unveils disaggregated inference demo with 2x speedup
By
–
The first disaggregated inference demo for AI agents is now live. At #COMPUTEX2026, SambaNova demonstrated premium inference running in production at VC2 — using NVIDIA B200 GPUs for prefill and SambaNova RDUs for decode. The result: 2x faster inference than B200-only
-
Full-load GPU limited to 220W with DFlash, DDTree optimizations
By
–
Looks like this under full-load btw Lots of juice to squeeze yet with DFlash / DDTree / Spec. Decoding / etc Also, power limiting the GPUs to 220w down from 440w as well (okay w/ leaving the perf. loss on the table given the heat / energy savings from that)