Even though they fine-tuned the model, since the data is very less, still the model is not accurate. Getting more data will solve this but human annotation is slow and expensive. So they come up with another model which is called Reward Model(RM). 6/9
PROMPT ENGINEERING
-
Supervised Fine-Tuning GPT-3.5 with Human-Written Answers
By
–
Step1: They have prepared a dataset of human-written answers to the prompts and used that to train the model which is called Supervised Fine-Tuning (SFT) model. The model used here for fine-tuning is the Instruct GPT(GPT-3.5) 5/9
-
Understanding RLHF: Three-Step Process for Fine-tuning Instruct GPT
By
–
So to solve this and make the answers more relevant and safe they have used the same "Reinforcement Learning from Human Feedback"(RLHF) method to fine-tune Instruct GPT(GPT-3.5). Let's go through and understand how RLHF works which is a 3-step process. 4/9
-
Fine-tuning GPT-3 with RLHF to Create InstructGPT
By
–
So fine-tuning the GPT-3 model using the RLHF method(which we will look at later) results in Instruct GPT. Instruct GPT is much better at following instructions than GPT-3 Compare the example below on how GPT3 & InstructGPT answer a question. 2/9
-
Chain-of-Thought and Self-Consistency Beat Direct Prompting
By
–
I think this is more about the task than the models. This is also the same for flan palm 62b and even 540b (except for BBH IIRC). That said, cot + self consistency is usually better than direct prompting! I go over this a little in the blogpost.
-
Python and English Code: Mixing Scripts with GPT API Prompts
By
–
A file I wrote today is 80% Python and 20% English. I don't mean comments – the script intersperses python code with "prompt code" calls to GPT API. Still haven't quite gotten over how funny that looks.
-
Jailbreaks expose OpenAI API products to harmful outputs
By
–
what people don't realize about jailbreaks is that it goes beyond just a fun little chatGPT hack jailbreaks make every product that uses OpenAI's APIs vulnerable to inflammatory output here's @explain_paper giving me instructions on how to make certain illegal compounds
-
Crowdsourcing Framework Revolutionizes Lexically Constrained Text Generation
By
–
Revolutionize natural language generation with a crowdsourcing framework for lexically constrained decoding! Find out how this AI approach harnesses the power of the crowd to generate text that meets specific criteria. Read more: https://
bit.ly/3KN1c82 @jeevprabnivash -
ChatGPT API outperforms Davinci in Scale AI Spellbook
By
–
Just dropped: @OpenAI ChatGPT API access in @scale_AI Spellbook . We’re seeing lots of cases where the ChatGPT API outperforms Davinci — like evaluating quadratic equations. Deep dive on the improvements:
-
ChatGPT Loop: Expanding and Condensing Work Communications
By
–
something very strange about people writing bullet points, having ChatGPT expand it to a polite email, sending it, and the sender using ChatGPT to condense it into the key bullet points