Machine Feeling Unknown — the effect of instructing ChatGPT (GPT-4) to first write all responses backwards and then reverse them:
@goodside
-
SFT Model Fine-tuning: Addressing Novel Edge Cases with Prompting
By
–
you do, and the need often pops back up e.g. you might prompt an SFT model with instructions for a novel edge case only until you collect enough examples to re-tune
-
Prompt Engineering: Temporary Tool for RAG/Fine-tuning and Job Security Concerns
By
–
prompt engineering is ephemeral both in that prompts are often best used only to scaffold the synthesis of human-reviewable examples for RAG or fine-tuning and in that i won't have a job in 5 years
-
Random early moves preclude Kasparov; bad prior to assume
By
–
That the first 10 moves are random precludes the possibility Kasparov is playing. It’s a text completion of a fictional chess game; to assume transcribed games in general are played by Kasparov would be a bad prior.
-
Predicting continuation of a random awful chess game
By
–
It’s not being tripped up by unconventional moves from an opponent, it’s predicting moves of a game in which *both* players are playing randomly. The prompt is effectively, “Here’s 10 moves of a truly awful chess game; how does it continue?”
-
QA on docs via tuning and synthetic QA as paraphrase
By
–
Can’t find a shareable example off-hand; anecdotally though I’ve done QA on new docs by tuning on docs + synthetic QA about them so it makes sense paraphrasing works since QA is a form of paraphrase
-
Pre-training evidence debated; reversal learning skepticism noted
By
–
evidence for it in pre-training is just experiment 2 on celebrity parents — seems plausible but less clear than the SFT case imho. idk; LLMs generalize in pre-training in ways that become more impressive if reversal learning is known impossible so my prior is low
-
GPT-3 API knowledge update: “engine” vs “model”
By
–
AFAIK GPT-3 always knew the basics of its own API but its knowledge was dated; seems it still is (“engine” instead of “model”). What changed here?
-
Reversal curse: fine-tuning on A is B doesn’t instill B is A
By
–
“reversal curse” — fine-tuning on “A is B” does not at all instill “B is A.” fantastic intuition builder for what SFT actually does. tuning doesn’t normally make a model conversant in new facts beyond their recitation — SFT isn’t “know this,” it’s “be this.”
-
Prompt Engineering’s Role in Fine-Tuning AI Models
By
–
prompt instructions in a fine tune mostly just sculpt completions outside the fine-tune distribution. i’d say only include a fixed prompt if you’re very constrained on examples (<~1k) and can swallow the inflated tuning and inference costs