My guess is this is InstructGPT/RLHF rather than anything in the pre-training corpus. Tuning implicitly makes it assume all questions are answerable — it sees all text as “ ” and Q/A is a subset of that.
@goodside
-
Reliable Knowledge from Parametric Memory in AI Remains an Open Problem
By
–
Yeah — getting reliable, truthful knowledge entirely from parametric memory seems like an open problem. The near-term solutions all involve grounding with queries to external resources.
-
AI Prompting Techniques and Hallucination Challenges
By
–
I spent some time trying to solve this problem through CoT/scratchpads but there were often secondary hallucinations, e.g. over-skepticism of non-trick questions. Nothing seemed to work as well as the zero-shot above.
-
New Spellbook AI Tool from Scale AI Enhances Prompt Engineering
By
–
Just dropped: Spellbook from @scale_AI. Great ideas in here — couples prompts with uploaded training data to automatically write k-shots, give evaluation metrics for prompt variants; also lets you turn prompts to spreadsheet functions https://t.co/hNGNXmGJtV
— Riley Goodside (@goodside) 9 novembre 2022Just dropped: Spellbook from @scale_AI
. Great ideas in here — couples prompts with uploaded training data to automatically write k-shots, give evaluation metrics for prompt variants; also lets you turn prompts to spreadsheet functions -
Prompt Engineering for AI Agents: Step-by-Step Operation Sequencing
By
–
The game is “least to most” — have it write out the most mechanical, reliably performed operations first. Then gradually move up to more interpretive and analytical steps, so it’s never asked to do anything multi-step in its head. The prompt should feel like an income tax form.
-
GPT-3 Tokenization Challenges and Model Limitations
By
–
+1 for this approach. GPT-3’s tokenization is a big gotcha. Model doesn’t see text as sequences of characters unless you add spacing/hyphens. It can’t even reliably tell you the last letter of a word without first rewriting it that way. Similar for counting length:
-
K-Shot Learning for Non-RLHF Tuned LLMs
By
–
I’ve never seen it work on anything other than RLHF-tuned GPT-3. I’ve wondered if you could k-shot a non-RLHF model with meta-examples of instruction templates followed by input/output pairs for each, but haven’t tried it.
-
Research on AI Example Benefits: Output Syntax and Label Space Learning
By
–
I think what Min et al. shows is that much of the benefit of examples is learning the output syntax and label space. Results aren’t as clean as this anecdote suggests by itself, e.g. OOD labels dramatically hurt performance.
-
Exploring Instruction Induction for Inferring AI Instruction Templates
By
–
Wonder how well it fares for longer instructions — I played around a while ago with using instruction induction to infer “instruction templates” from examples and it seemed to work:
-
Experimental Optimization of AI Instruction Prompts
By
–
Cool extension of Honovich et al.’s “instruction induction”, experimentally optimizing to find the best instruction prompt for a given set of task examples: https://t.co/h4Rd2A9JwY
— Riley Goodside (@goodside) 4 novembre 2022Cool extension of Honovich et al.’s “instruction induction”, experimentally optimizing to find the best instruction prompt for a given set of task examples: