We ran about 50 steps on the CS-3. Saw nice loss and stable training curves
RESEARCH
-
Cerebras MemoryX: 55TB External Memory for Trillion Parameters
By
–
Cerebras Wafer Scale Cluster uses unique, terabyte-scale external memory device called MemoryX to store model weights. For Sandia’s run, Cerebras configured a 55 terabyte MemoryX device – enough to comfortably store 1T parameters and optimizer states.
-
Cerebras CS-3 trains 1 trillion parameter model efficiently
By
–
Cerebras Systems + Sandia National Labs have demonstrated training of a 1 trillion parameter model on a single CS-3 system (!) This is ~1% the footprint & power of an equivalent GPU cluster.
-
Team Wins Best Paper Award at NeurIPS Conference
By
–
We're very proud of our team's contributions to cutting-edge research. Congratulations @max_nlp and collaborators for winning a best paper award @NeurIPSConf
! -
AI Models Confidently Make Mistakes Despite External Context
By
–
AI systems often gather external info as context when answering a particular query. Even w/this relevant info, models can make mistakes w/seemingly high doses of confidence.
-
MIT ContextCite tool improves chatbot answer reliability verification
By
–
How can we really know if a chatbot is giving a reliable answer? MIT CSAIL’s "ContextCite" tool can ID the parts of external context used to generate any particular statement from a language model, improving trust by helping users easily verify the statement:
-
Latent Space LIVE keynote 2024 AI year review at NeurIPS
By
–
Presenting Latent Space LIVE! https://
youtube.com/watch?v=wT636T
HdZZo
… LIVE from the beautiful Wosk Centre at NeurIPS! kicking off with 2024 Year in Review keynote from @saranormous and @pranavreddy ! -
Data Scaling Limitations in AI Model Improvement Questioned
By
–
Shared some hot takes with @reckless on Decoder this week (not all of them AGI-related!), and one of the most important: I don't think data will be a limitation on model improvement and scaling anytime soon. Why I think we still have big gains ahead: 1. The more computation you
-
Google achieves top AI rankings with Gemini 2.0 and Project Astra
By
–
It's been a good month, #1 on Lmsys chatbot arena, Imagen3, Gemini 2.0, Project Astra, Project Mariner, and Jules. Congrats Google! 😀
-
Multimodal Models Unlock Time-Series Data Analysis Power
By
–
Unlocking the power of time-series data with multimodal models https://
bit.ly/4eXGIVr
#AI #MachineLearning #DeepLearning #LLMs #DataScience