In the future, importing Keras from TF (`tf.keras`) will give you multi-backend Keras. However we will provide a way to revert back to the legacy tf-only Keras (exact same setup as before), so as to enable folks to upgrade their TF version without worrying about minor breakages.
MACHINE LEARNING
-
Unity Catalog Shared Access Mode Features Announcement
By
–
What’s new for #UnityCatalog clusters in shared access mode: Cluster libraries Python & Pandas UDFs Single-node ML See how these features can help you seamlessly move workloads to shared clusters and more https://
bit.ly/3Pdp85c -
BTLM-3B Outperforms Larger 7B Models in Long Context
By
–
The paper explores long context performance in detail. We found BTLM-3B-8K outperforms other 7B-8K models despite being trained with less than a fifth the pretraining compute and less than half the size.
-
ALiBi Enables Context Extrapolation Beyond Training Length
By
–
ALiBi allows our model trained on 8192 context length to extrapolate well up to 9216 tokens out of the box. We also show that ALiBi models only achieve near-perfect extrapolation when they are severely undertrained (<1 tokens/parameter).
-
BTLM: World’s Most Accurate 3B Parameter Open Source Model
By
–
BTLM is the world’s most accurate 3B parameter model, with performance that rivals many 7B models. It’s fully open source and has been downloaded >1M times.
-
LLM Training Techniques: 2.86x Compute Efficiency Gains
By
–
There are so many LLM training tricks but which ones should you use? We quantify the effects of techniques like SwiGLU, ALiBi, etc and show how using them in conjunction can achieve the same loss with 2.86x less pretraining compute or 1.74x less parameters.
-
BTLM-3B-8K: Distilling SOTA LLM Training Recipe
By
–
We just dropped the BTLM-3B-8K paper on arXiv! It distills our recipe for training SOTA LLMs:
– Extensively deduplicated dataset (SlimPajama)
– Hyperparameter search using muP
– Variable sequence length training + ALiBi
– Aggressive LR decay https://
arxiv.org/abs/2309.11568 -
AI Model Diagnoses Medical Images from 200,000 Cancer Cases
By
–
From 200,000+ anonymized diagnostic images of cancers and other diseases on Twitter, scholars have built a powerful AI model that can analyze and diagnose previously unseen medical images.
-
PyDataset in Keras Core: New Sequence Class for Data Loading
By
–
There's a new Sequence class in Keras Core, it's called PyDataset. It's basically the same functionality (in particular parallel loading), no major improvement. I would still recommend using http://
tf.data for anything performance critical. -
Emergent Abilities in LLMs: Evidence of In-Context Learning
By
–
Are Emergent Abilities in Large Language Models just In-Context Learning? ***********
Spoiler: YES Through a series of over 1,000 experiments, we provide compelling evidence: http://
arxiv.org/abs/2309.01809 Our results allay safety concerns regarding latent hazardous