Not sure how it will play out, but it means that generations will be very fast for such a big model, as only a small portion of params will be active at a time.
SYSTEMS
-
New Book on AI Engineering with Foundation Models
By
–
I’m excited to share that I’m working on a new book about building applications with foundation models! AI Engineering builds upon Machine Learning Systems Design, but with a focus on large scale, ready made models. The book covers: – The new AI stack (e.g. how it differs from
-
State Management for AI Agents: Side Effects and UX
By
–
state management is hard! we've been thinking a lot along the lines of staged side effects, io monads, and how to represent real world state changes as well. doing this right, with the right ux, will unlock a lot more user comfort with agents.
-
Evaluating Speed Cost and Offline AI Performance Metrics
By
–
Speed, cost and the ability to run offline on a personal device are the things I care most about I still don't have a great answer for how to best evaluate these things but I'm researching that at the moment
-
Faster AI Inference Enables More Thinking Same Time
By
–
The model will output the same result, no matter the speed. What faster inference enables is more thinking in the same amount of time.
-
Edge AI Adoption: Real-Time Decision-Making Through Model Optimization
By
–
The Promise of Edge AI and Approaches for Effective Adoption: Organizations are adopting edge AI for real-time decision-making using efficient and cost-effective methods such as model quantization, multimodal databases, and distributed inferencing. https://
kdnuggets.com/the-promise-of
-edge-ai-and-approaches-for-effective-adoption?utm_source=dlvr.it&utm_medium=twitter&utm_campaign=the-promise-of-edge-ai-and-approaches-for-effective-adoption
… -
State-Space Models Express Power Limited to TC0 Complexity
By
–
6/ The Illusion of State in State-Space Models – investigates the expressive power of state space models (SSMs) and reveals that it is limited similar to transformers in that SSMs cannot express computation outside the complexity class 𝖳𝖢^0…
-
AI-Driven Supply Chain Risk Management for Critical Infrastructure
By
–
Discover the powerful synergy between @aDolus_Inc OT systems analysis & Exiger's AI-driven supply chain risk mgmt in our latest interview. Learn how this partnership benefits end users, particularly in critical infrastructure sectors https://t.co/3IidesPRNn #sponsored #adolus_ics pic.twitter.com/dMjBl9Ei6Q
— Lucian Fogoros (@fogoros) 20 avril 2024Discover the powerful synergy between @aDolus_Inc OT systems analysis & Exiger's AI-driven supply chain risk mgmt in our latest interview. Learn how this partnership benefits end users, particularly in critical infrastructure sectors https://
ow.ly/O5Cx50QPqqH #sponsored #adolus_ics -
RAG vs Large Context Windows: Enterprise AI Strategy
By
–
As AI evolves, the choice isn't just between supermassive context windows and retrieval-augmented generation (RAG)—it's about leveraging RAG for enterprise success. Despite the perks of larger context windows, they bring increased costs and latency. RAG, however, remains
-
M3 Max 8B model optimization for multi-agent orchestration systems
By
–
I have an M3 Max with 64GB. You can use the 8B model if you don't have a lot of VRAM. What I realize is that the 8B model works really well as a subagent. As long as the Orchestrator and the Refiner are the 70GB models, the whole thing works really well.