Early benchmark evaluations of Grok-0 (33B parameters of training data) show promise against LLaMa 70B, though still behind GPT-4, which has around 5x the amount of training data. Grok-1 has yet to be peer evaluated.
LLMS
-
FP8 Low-Precision Training Improves LLM Efficiency Without Hyperparameter Changes
By
–
10/ FP8-LM – finds that when training FP8 LLMs most variables, such as gradients and optimizer states, in LLM training, can employ low-precision data formats without compromising model accuracy and requiring no changes to hyper-parameter.
-
Enhancing LLMs Understanding through Emotional Stimuli Research
By
–
9/ Enhancing LLMs by Emotion Stimuli – explores the ability of LLMs to understand emotional stimuli; conducts automatic experiments on 45 tasks using various LLMs, including Flan-T5-Large, Vicuna, Llama 2, BLOOM, ChatGPT, and GPT-4.
-
Efficient Context Window Extension Method for Large Language Models
By
–
5/ Efficient Context Window Extension of LLMs – proposes a compute-efficient method for efficiently extending the context window of LLMs beyond what it was pretrained on; models of up to 128K context length reproduced.
-
LLMs Applied to Industrial Chip Design and Optimization
By
–
4/ LLMs for Chip Design – proposes using LLMs for industrial chip design by leveraging domain adaptation techniques; evaluates different applications for chip design such as assistant chatbot, electronic design automation, and bug summarization.
-
Comprehensive Survey on LLM Evaluation Methods and Datasets
By
–
2/ Evaluating LLMs – a comprehensive survey (100+ pages) on evaluating LLMs, including discussions about the different types of evaluations, datasets, techniques, and more.
-
Top ML Papers: LLMs, AlphaFold, and Advanced Techniques
By
–
Top ML Papers of the Week (Oct 30 – Nov 5): – LLMs for Chip Design
– Battle of the Backbones
– Next Generation AlphaFold
– Symmetry in Machine Learning
– Enhancing LLMs by Emotion Stimuli
– Efficient Context Window Extension of LLMs
… -
Emergence of Specialized Domain-Specific LLMs in Finance and Law
By
–
In some very specific cases, we will also see some very specialized LLMs emerge – e.g. financeLLM or LegalLLM. These domain-specific LLMs that may require a lot of custom training, RLHF, and fine-tuning. /16
-
LLMOps Platforms Enable Flexible LLM Selection for Enterprises
By
–
Companies will use LLMOps platforms such as Abacus to automate an end-to-end workflow. These platforms will offer a combination of open-source LLMs and closed-sourced APIs. Companies should be free to pick and choose an LLM based on cost, performance, and time to market. /15
-
Smaller LLMs with RAG as cost-effective alternative to GPT-4
By
–
Smaller and more efficient LLMs that have reasonably good reasoning capabilities can be fine-turned or complemented with RAG and incorporated into the workflow to automate these use cases. Using GPT-4 or some other very large LLM will become cost-prohibitive in these cases. /14