This week, HAI faculty affiliate @sanmikoyejo joined @CoryBooker and other visionaries, activists, and leaders at the @CBCFInc
’s #ALC52. An important conversation to ensure that we’re shaping the future of AI with equity at its core.
INNOVATION
-

AI Equity Leaders Shape Future at Congressional Black Caucus Forum
By
–
-

Neural Foundations of Mental Simulation in Dynamic Scenes
By
–
Neural Foundations of Mental Simulation: Future Prediction of Latent Representations on Dynamic Scenes Nayebi et al.: https://
arxiv.org/abs/2305.11772 #ArtificialIntelligence #DeepLearning #MachineLearning -
Natural Language Interfaces Enable Enterprise AI Data Conversations
By
–
For years I have believed the future of human computer interactions would be based on conversations that enable humans to use their natural language. Now thanks to @vectara
, developers can make this a reality over just about any enterprise data. -

OpenAI Cookbook Gets New Design for Better Discovery
By
–
New skin for the OpenAI cookbook, our open-source collection of examples and guides for building with the OpenAI API. Easier to use and discover than ever before: https://
cookbook.openai.com Great work led by @simonpfish
. -

BTLM-3B Outperforms Larger 7B Models in Long Context
By
–
The paper explores long context performance in detail. We found BTLM-3B-8K outperforms other 7B-8K models despite being trained with less than a fifth the pretraining compute and less than half the size.
-

LLM Training Techniques: 2.86x Compute Efficiency Gains
By
–
There are so many LLM training tricks but which ones should you use? We quantify the effects of techniques like SwiGLU, ALiBi, etc and show how using them in conjunction can achieve the same loss with 2.86x less pretraining compute or 1.74x less parameters.
-

BTLM-3B-8K: Distilling SOTA LLM Training Recipe
By
–
We just dropped the BTLM-3B-8K paper on arXiv! It distills our recipe for training SOTA LLMs:
– Extensively deduplicated dataset (SlimPajama)
– Hyperparameter search using muP
– Variable sequence length training + ALiBi
– Aggressive LR decay https://
arxiv.org/abs/2309.11568 -

Copilot’s New Features Boost Creativity and Productivity
By
–
Boost your creativity and productivity with personalized answers and Bing Image Creator. Discover these and other new features here: https://
msft.it/60199ihBc -
OpenAI Launches Cookbook at New Home for Developers
By
–
Awesome news for @OpenAI devs: the cookbook has a new home, https://
cookbook.openai.com Thousands of folks show up everyday to use the cookbook, excited for this content to be more accessible. Big s/o to @simonpfish for driving this!