btw if you want to chat about your RLHF strategy with us @scale_AI — reach out! DMs are open source on OpenAI:
LLMS
-
Flash Attention and PaLM Talks: Direct Video Pointers
By
–
Here are direct pointers for flash attention and PaLM talks: Flash Attention: https://
youtube.com/watch?v=gMOAud
7hZg4
… PaLM – Pathways Language Model and Model Scaling: https://
youtube.com/watch?v=CV_eBV
wzOaw
… Upcoming talks: https://
mlsys.stanford.edu -
Stanford CS 324 Foundation Models Course Partnership
By
–
Those talks are hosted in partnership with Stanford CS 324 – Advances in Foundation Models. More on CS 324 – Advances in Foundation Models: https://
stanford-cs324.github.io/winter2023/ -
Stanford MLsys Talks on Flash Attention and PaLM
By
–
Stanford MLsys – Talks on Flash Attention and PaLM ML systems talks hosted by Stanford MLsys are always amazing. Stanford MLsys is currently hosting talks around foundational models. Two most notable ones so far are Flash Attention and PaLM(Pathways LM). https://
youtube.com/@StanfordMLSys
Seminars/streams
… -
Adding ‘Exclude special tokens’ changes AI classifier output
By
–
Found that adding "Exclude the use of special tokens" at the end of each ChatGPT prompt has consistently changed the AI Text Classifier output from "likely AI generated" to "possibly AI generated" Either a coincidence, or special tokens are watermarking ChatGPT outputs.
-
Flan-T5 Outperforms T5 in Finetuning Efficiency
By
–
Key takeaway: finetuning Flan-T5 is better and more compute-efficient than finetuning T5. In other words, Flan-T5 > T5 for every real scenario I can think of. Don't use the pre-trained checkpoint—always finetune!
-
Google releases instruction tuning collection for improved language model reasoning
By
–
Today we’re releasing a new collection of tasks, templates and methods for instruction tuning of #ML models. Training on this collection can enable language models to reason more competently over arbitrary, unseen tasks. Learn all about it at: https://
goo.gle/3XWlqjc -
Transformer Variations: Major Refactoring Update Three Years Later
By
–
Cannot believe it is almost 3 years since my 2020 post on variations of Transformer. I spent some time and did a big refactoring of that old post with new section structure and new papers. Still missing a few items tho, will add them in slowly:
-
OpenAI Maintains Free ChatGPT Access While Launching Paid Subscription
By
–
Access to ChatGPT would still be available for FREE By OpenAI – "We love our free users and will continue to offer free access to ChatGPT. By offering this subscription pricing, we will be able to help support free access availability to as many people as possible."
-
OpenAI Launches ChatGPT Plus Paid Subscription Service
By
–
Breaking OpenAI just launched ChatGPT plus, a paid subscription plan for $20/month. It will come with the following benefits: – General access to ChatGPT, even during peak times
– Faster response times
– Priority access to new features and improvements