The tricky part: naïvely casting BF16 group scales to FP8 dropped the quality. Our fix: quantize scales per-channel (outer vector scaling) + rescale by 1/8 to avoid FP8 clipping. Result: >99.5% of W4A16 accuracy recovered on Command A & Cohere MoE. Paired with a CUTLASS
CODE
-
W4A8 Inference Production-Ready Integration in vLLM
By
–
Excited to share our work on production-ready W4A8 inference, now integrated in vLLM! By combining 4-bit weights (low memory) with 8-bit activations (high compute), we hit the sweet spot for both decoding and prefill — up to 58% faster TTFT and 45% faster TPOT vs W4A16 on Hopper.
-
First Notebook Competition with marimo to Implement AI Papers
By
–
Looking for a weekend project that is both challenging and rewarding? Enter our first notebook competition with @marimo_io to implement AI papers. You also got 5 days left if you want to win a Mac Mini + over $1K in prizes! Full details found below
-
Plugin Architecture for Multiple AI Providers Integration
By
–
It’s all plugins – just like we have plugins for alibaba, tencent, kimi, minimax, anthropic and many others.
-
Belief States Framework Brings Back Advanced Agent Reasoning
By
–
Very nice way to bring back belief states
-
HuggingFace CLI Invocations by Coding Agents
By
–
It’s only invocations of the hf CLI by the coding agents. So it’s really only a partial data point.
-
Coding Agents Use HuggingFace CLI for Invocations
By
–
It’s only invocations of the hf CLI by the coding agents. So it’s really only a partial data point.
-
AngyPenguin’s HuggingFace Space Still Accessible
By
–
I think @angrypenguinPNG
’s old HuggingFace space might still work -
Shopify’s AI-Native Engineering: 100% Adoption and SimGym
By
–
🆕 Shopify's AI-Native Engineering: 100% adoption, unlimited tokens, Tangle, Tangent, & SimGymhttps://t.co/5MOZRGrSbO@Shopify CTO @MParakhin explains why near-universal AI adoption is changing how Shopify builds, how SimGym uses Shopify’s scale and historical data to simulate… pic.twitter.com/XLKKVbJ5QX
— Latent.Space (@latentspacepod) 22 avril 2026Shopify's AI-Native Engineering: 100% adoption, unlimited tokens, Tangle, Tangent, & SimGym https://
latent.space/p/shopify @Shopify CTO @MParakhin explains why near-universal AI adoption is changing how Shopify builds, how SimGym uses Shopify’s scale and historical data to simulate