Effective Long-Context Scaling of Foundation Models https://
bit.ly/3ER36jT Another piece of research that helps us build engaging conversational experiences for our AIs and the Meta AI assistant.
LLMS
-
Effective Long-Context Scaling of Foundation Models Research
By
–
-
Code Llama: Open Foundation Models for Code Generation
By
–
Code Llama: Open Foundation Models for Code https://
bit.ly/3RCyvhr A version of Llama 2, fine-tuned for generating and discussing code. -
Language Model Evaluation Methods: From Casual Testing to Rigorous Benchmarks
By
–
Just a joke, don’t take this meme too seriously and pls do rigorous evals 🙂 Explanation:
– Left: The simplest way to evaluate a language model is to play with it for 15 minutes. This is not scientific at all.
– Middle: The more systematic way is to create a diverse set of -
Language Model Evaluation: Challenges and Solutions After 5 Months
By
–
Large language models are notoriously hard to evaluate because (1) they are highly multi-task, (2) they generate long completions, and (3) grading is subjective. After spending ~5 months rigorously working on how to do language model evals, this is my verdict:
-
Pair Programming with LLMs: Emerging Best Practices for Developers
By
–
Many AI-savvy programmers are now coding very differently than before, by using LLMs to help with their work. You’ll learn these emerging best practices in “Pair Programming with a Large Language Model,” by Google's @lmoroney.
— Andrew Ng (@AndrewYNg) 27 septembre 2023
This short course covers using LLMs to simplify and… pic.twitter.com/6IahhCQPsNMany AI-savvy programmers are now coding very differently than before, by using LLMs to help with their work. You’ll learn these emerging best practices in “Pair Programming with a Large Language Model,” by Google's @lmoroney
. This short course covers using LLMs to simplify and -
ChatGPT Browsing Returns with Major Behind Scenes Improvements
By
–
ChatGPT browsing is back! 🔥
— Logan Kilpatrick (@OfficialLoganK) 27 septembre 2023
There’s been a ton of behind the scenes improvements to browsing and I’m super excited for people to get their hands on it again.
Knowledge cutoff no more! pic.twitter.com/BSBr2Jzsk6ChatGPT browsing is back! There’s been a ton of behind the scenes improvements to browsing and I’m super excited for people to get their hands on it again. Knowledge cutoff no more!
-
ChatGPT Now Browses Internet for Current Information
By
–
ChatGPT can now browse the internet to provide you with current and authoritative information, complete with direct links to sources. It is no longer limited to data before September 2021. pic.twitter.com/pyj8a9HWkB
— OpenAI (@OpenAI) 27 septembre 2023ChatGPT can now browse the internet to provide you with current and authoritative information, complete with direct links to sources. It is no longer limited to data before September 2021.
-
Exploring Hidden Door’s AI Technology Behind the Scenes
By
–
Taking a peek behind Hidden Door's AI https://
bit.ly/3PIode9 #AI #MachineLearning #DeepLearning #LLMs #DataScience -
Thanks to compute providers and partners for model release support
By
–
Huge thanks to our compute providers @CoreWeave and the EuroHPC, @tri_dao and @d_haziza for their help with FlashAttention and xFormers, and @huggingface
, vLLM, @skypilot_org
, FastChat for their help and support with this release. -
GQA and Sliding Window Optimization for Efficient Decoding
By
–
We trained it with GQA and a sliding window of 4096 tokens, resulting in constant cache size and a linear decoding speed. Our changes to FlashAttention v2 and xFormers to support sliding window are available to the community.