Some setup notes (cc @EthanJPerez
) – We used the exact 2-shot prompt for Quote Repetition, which is already U-shaped for Gopher/Chinchilla – We used fewer shots for Hindsight – We did few-shot instead of 0-shot for Negation QA – We also showed inverse scaling up to PaLM 62B
PROMPT ENGINEERING
-
Few-shot Prompt Setup Notes for Scaling Law Experiments
By
–
-
U-shaped Scaling and the Limitations of Inverse Scaling Tasks
By
–
Implications: 1. U-shaped scaling means that inverse scaling may not hold when extrapolated to larger models. 2. The term “inverse scaling task” is underspecified. A task can be inverse scaling for one type of prompting and positive or U-shaped for another type of prompting.
-
CoT Prompting Defends Against Inverse Scaling in Math Tasks
By
–
Second, we show that CoT prompting can defend against inverse scaling. For instance, CoT prompting achieves 100% on 7 out of 8 subtasks for Redefine Math.
-
Exploring Instruction Induction for Inferring AI Instruction Templates
By
–
Wonder how well it fares for longer instructions — I played around a while ago with using instruction induction to infer “instruction templates” from examples and it seemed to work:
-
Experimental Optimization of AI Instruction Prompts
By
–
Cool extension of Honovich et al.’s “instruction induction”, experimentally optimizing to find the best instruction prompt for a given set of task examples: https://t.co/h4Rd2A9JwY
— Riley Goodside (@goodside) 4 novembre 2022Cool extension of Honovich et al.’s “instruction induction”, experimentally optimizing to find the best instruction prompt for a given set of task examples:
-
Large Language Models Achieve Human-Level Prompt Engineering
By
–
Large Language Models Are Human-Level Prompt Engineers Zhou et al.: https://
arxiv.org/abs/2211.01910 #MachineLearning #DeepLearning #ArtificialIntelligence -
Powerful Tool for K-Shot Datasets and Prompt Chaining with Collapsible Tree UI
By
–
@dust4ai takes some work to wrap your head around, but it's very powerful — gives a collapsible tree UI for representing k-shot example datasets, prompt templates, and prompt chaining with intermediate JS code. Replaces a lot of code around prompt APIs. http://
dust.tt -
Langchain Python Package Implements Advanced LLM Prompting Techniques
By
–
@hwchase17
's Python package Langchain implements and lets you easily compose many published LLM prompting techniques. Implements self-asking, web search, REPL math, and several of my own prompts. -
EveryPrompt Extends Playground for Prompt Variables and API Integration
By
–
@everyprompt extends Playground in a similar way: Putting variables in prompts and giving you a single button to go from prompt to API. Has nice developer-oriented touches in the UI too — e.g. displaying invisible chars as ghosts. https://
everyprompt.com