To support efficient serving of Jamba-1.5-Large, we developed a novel quantization technique – ExpertsInt8. We quantize the MoE and MLP weights to INT8 in order to store them, and dequantize them back to BF16 before the actual computation. This technique is both very fast and
CODE
-
LM Studio v0.3 Released with Improved User Interface
By
–
Also enjoying the v0.3 of @LMStudioAI
, the new UI is slick -
Stable FP8 Training Recipe for Llama Models
By
–
Summary of stable FP8 training recipe for Llama models:
1. Clip learning rate when 2nd moment estimator is outdated by coming spike – see https://
arxiv.org/abs/2304.13013
2. SmoothQuant: smooths activation outliers by migrating the quantization difficulty to weights – see -
Scalable Autoregressive Image Generation with Mamba Architecture
By
–
Scalable Autoregressive Image Generation with Mamba discuss: https://
huggingface.co/papers/2408.12
245
… model: https://
huggingface.co/hp-l33/aim We introduce AiM, an autoregressive (AR) image generative model based on Mamba architecture. AiM employs Mamba, a novel state-space model characterized by its -
Building Agentic Workflows with Multiple Function Support
By
–
Here’s what is supported! Imagine being able to quickly build agentic workflows that leverage any of these functions!
-
Claude.ai Enables LaTeX Rendering for Conversations
By
–
Enable LaTeX rendering for new conversations on http://
claude.ai by toggling on the feature preview here: https://
claude.ai/new?fp=1. -
Claude Adds LaTeX Rendering Support for Mathematical Equations
By
–
We've added support for LaTeX rendering as a feature preview. Claude can now display mathematical equations and expressions in a consistent format.
-
Conda Alternatives for Managing PyTorch and DeepSpeed Dependencies
By
–
please suggest conda alternatives. needs to manage torch, deepspeed, etc without fuss.
-
Cursor Raises $60M for AI-Powered Coding Platform
By
–
We've raised $60M from Andreessen Horowitz, Jeff Dean, John Schulman, Noam Brown, and the founders of Stripe and Github. Cursor has become recognized as the best way to code with AI, powered by an ensemble of custom and frontier models, delightful editing, and petabyte-scale
-
Build Voice AI Products with AssemblyAI Speech-to-Text API
By
–
Weekly Spotlight: Build voice-driven AI products with just 10 lines of code using @AssemblyAI
's top-notch speech-to-text API. High accuracy, low latency, and multilingual support! #LowCode #AI #API #STT