HARDWARE

–

21 June 2026 20h55

Why do I focus on Inference Engines/Software Stacks for your hardware? – 2x RTX 3090s: ~14.5 tok/s → ~64 tok/s moving to vLLM w/ TP=2 – RTX PRO 6000: ~32 tok/s → ~110 tok/s moving to Sglang So: – CUDA/2+ GPUs: ExLlamaV3/vLLM/Sglang > llama.cpp – Edge: llama.cpp > Ollama

→ View original post on X — @scobleizer

Claude AI self-optimizes upon discovering voice harness

By

@scobleizer

–

21 June 2026 9h36

I met a guy last night building a next-level voice harness. He hooked Claude up to it and it realized quickly the way the voice model worked and optimized itself for that use.

The bible for running LLMs locally now free online

By

–

21 June 2026 4h56

DROP EVERYTHING The bible for running LLMs locally is now available online to read for free Covers what to use on – Laptop / edge / odd hardware
– Mac-first workflows
– Single RTX GPUs
– 2-4+ NVIDIA / CUDA GPUs
– General production serving
– Long-context / MoE / routing
–

Stop hardware cost to token calculations, models improve, prices rise

By

–

21 June 2026 2h42

Can we stop doing hardware cost to token generation calculations on the timeline please? If you haven't noticed, models keep getting better & more efficient, and hardware prices keep going up

→ View original post on X — @maximelabonne

ColBERT outperforms on CPU with low latency for embeddings

By

@maximelabonne

–

20 June 2026 23h45

I'm talking about individual descriptions used for embeddings. It doesn't need to be particularly long for late interaction to perform better. The tradeoff really depends on the use case. In this case, even on a cheap CPU, the latency is so low that ColBERT just works better!

20 June 2026

Luke Alonso uploaded NVFP4 of GLM 5.2, 467GB on 4 DGX Sparks

By

–

20 June 2026 23h00

Luke Alonso has uploaded an NVFP4 of GLM 5.2 467GB, would fit on 4x DGX Sparks (~$20k)