And it outperforms vLLM when deploying DeepSeek-R1-671B on an A800 (40GB) cluster.
OPEN SOURCE
-
Chitu: High-Performance Open-Source LLM Inference Framework
By
–
Yet another open-source project from China: Chitu! A high-performance inference framework for LLMs, designed for efficiency, flexibility, and availability. Chitu supports various mainstream models, including DeepSeek, the LLaMA series, Mixtral, and more.
-
Automating Web Workflows with LLMs at FOSSASIA Bangkok
By
–
Still weakness but all ready for my talk today about automating web workflows using LLMs at @fossasia Bangkok. 🙂 pic.twitter.com/2gYkPHz5RL
— Saurav Jain (Open Source + Communities) (@Sauain) 14 mars 2025Still weakness but all ready for my talk today about automating web workflows using LLMs at @fossasia Bangkok. 🙂
-
Speculation on Model Distillation and Gemma-Gemini Similarities
By
–
Good catch. Yes, maybe that's distilled down from a larger model they didn't open source. In that context, I am also curious how similar Gemma and Gemini models are.
-
Google Releases Gemma 3 and Gemini 2.0 Flash
By
–
AI News from the past week that I've found interesting. Let me know about (and link me up to) anything that I missed this week so I can be sure to cover it in my breakdown video… (March 7th – 13th)
– Google Releases open-weight Gemma 3 model
– Google releases Gemini 2.0 Flash -
SambaNova Launches Open-Source AI Agents Community Initiative
By
–
Bring Your Own Agents AI Style We’re all about open-source contributions! Deploy your SambaNova-powered agent, share your breakthroughs, tag us, & stay tuned for contests & prizes! Details here https://
sambanova.ai/blog/open-sour
ce-deep-research-agents
… #AIGameOn -
MIT’s Exo 2: High-Performance Programming Language Reduces Code Requirements
By
–
A new programming language from MIT could enable high-performance coding that can compete w/state-of-the-art libraries w/a few hundred lines of code, instead of tens or hundreds of thousands. Called "Exo 2," the approach enables reusable scheduling libraries external to
-
AI Agent Dev Meetup Silicon Valley: Fast Inference and Open Source
By
–
Guess who's back? The team is going to be at the hottest AI Agent dev meetup in Silicon Valley once again! Fast Inference Open Source AI Agentic AI … we're talking about it all! Join us https://
lu.ma/rm5l38en -
Llama Models Power India’s First Open Source Audio Language Model
By
–
Llama models were used to develop India's first open source audio language model — Shuka v1 https://
go.fb.me/4bbs7t -
Anthropic API Updates for 2025
By
–
Anthropic got some API updates – Cache-aware rate limits
– Simpler prompt caching
– Token-efficient tool use