Quick question, what's your expected minimum token per second when using a local LLM?
SOFTWARE
-
MPC as API Bridge Between LLMs and Software Tools
By
–
Yes thats the beauty of MPC, its like an API between the LLM and Tools (apps, services, etc) This is the future of how we will interact with software
-
SuperWhisper: Voice AI Tool Accelerates Productivity and Workflow
By
–
Together with something like https://
superwhisper.com and you can just talk your way there super fast -
Chitu Matches vLLM Performance on H20 DeepSeek-R1
By
–
On an H20 (96GB) cluster, Chitu performs comparably to vLLM when deploying DeepSeek-R1-671B.
-
Chitu: High-Performance Open-Source LLM Inference Framework
By
–
Yet another open-source project from China: Chitu! A high-performance inference framework for LLMs, designed for efficiency, flexibility, and availability. Chitu supports various mainstream models, including DeepSeek, the LLaMA series, Mixtral, and more.
-
Ten Thousand Pi Digits Visualized in MATLAB
By
–
Ten thousand digits of pi visualized in MATLAB 😮 Happy Pi Day! pic.twitter.com/cA4CVQgGcN
— MathWorks (@MathWorks) 14 mars 2025Ten thousand digits of pi visualized in MATLAB Happy Pi Day!
-
ChatGPT Work with Apps Now Available on macOS
By
–
Work with Apps on macOS is now available to everyone, including Enterprise, Edu, and Free users. ChatGPT can read and edit content in your coding apps, bringing you smarter answers tailored to your work and helping you stay in flow. But don't forget to update the app!
-
MIT’s Exo 2: High-Performance Programming Language Reduces Code Requirements
By
–
A new programming language from MIT could enable high-performance coding that can compete w/state-of-the-art libraries w/a few hundred lines of code, instead of tens or hundreds of thousands. Called "Exo 2," the approach enables reusable scheduling libraries external to
-
All Wan2.1 Models Now Available in One Place on Replicate
By
–
You can now find all Wan2.1 models in one place on Replicate.