Has anyone tried loading two models into memory w/ llama cpp? Like an 8bit Mistral 7B and a 4bit Mixtral simultaneously.
COMPUTING
-
CPU Management System Control Claims Lack Evidence
By
–
"they own the management system embedded in any remotely modern CPU" is a declarative statement that is not backed by evidence. Their having *wanted* such power/control is not proof that they have it.
-
Tenstorrent Delivers High-Performance RISC-V AI Chiplets
By
–
“Tenstorrent is delivering highly customized, high-performance AI and #riscv chiplets”, said CEO of @MosChipTech
. “In testing simulations done thus far, we’ve determined that Tenstorrent’s RISC-V core is one of the highest performing CPUs we’ve seen.” -
VMware Broadcom NVIDIA Partner AI Data Center Edge Solutions
By
–
Maintain privacy and control of your data while bringing the power of #AI to your data centers and the edge. At #GTC24, learn how @VMware by @Broadcom and NVIDIA are working together to bring flexibility and choice to your many use cases. Register now. https://
nvda.ws/432dXCM -
LLM Capabilities Growing Several Times Faster Than Moore’s Law
By
–
Here's a good estimate of how fast the capabilities of LLMs have been growing: several times as fast as Moore's Law! The compute needed to achieve the same outcome halving every 5 to 14 months, with no sign of slowing. Most gains are from bigger scale. https://
arxiv.org/abs/2403.05812 -
Web Inventor Predicts VR and Spatial Computing Transform Internet
By
–
World Wide Web inventor predicts #VR and spatial computing will transform the internet https://
cointelegraph.com/news/world-wid
e-web-inventor-predicts-vr-and-spatial-computing-web3-metaverse-transform-internet
… via @cointelegraph -
Meta Infrastructure Scheduler: Open-Source Limitations and Market Analysis
By
–
its a bigger lift to open-source that, very specific to a lot of how we build Meta Infra.
Also, there isn't a good addressable market, SLURM already exists and is a good HPC scheduler. -
Public Sector AI Investment Moonshot Called for by Stanford HAI
By
–
AI requires vast amounts of computing power and data that are becoming increasingly out of reach for academia. @StanfordHAI
’s @drfeifei is one of the leading voices calling for a “moonshot investment” in public-sector AI. https://
wapo.st/4a6IVvN -
Startup AI Success Requires Absorption Into Big Tech Infrastructure
By
–
It's that what OAI x Microsoft heralded? Mistral x Microsoft? I don't see a future where startups consolidate w/each other, but I do see the only path to startup "success" being absorption of some kind into Big Tech infra providers, who offer compute + market access.
-
Custom Scheduler vs SLURM for Workload Management
By
–
we have our own scheduler, but I'd assume SLURM would be a good choice as well.