The sense used in the InstructGPT paper is good — a model is aligned when it does what its designers want. Instruction tuning is the canonical form of LLM alignment, but earlier methods like filtering pre-train data of undesired content count too.
@goodside
-
Differentiating AI Jailbreaks from Prompt Injection Attacks
By
–
What you’re describing is a jailbreak, not a prompt injection. Prompt injection is when data is misinterpreted as part of the prompt instructions (against the intention of the prompter). A jailbreak is when the prompter bypasses safety policies of the model.
-
LLM Prompt Injection via Invisible Instructions in Pasted Text
By
–
PoC: LLM prompt injection via invisible instructions in pasted text
-
Focus on Red Teaming and Adversarial Data for LLM Post-Training
By
–
Red teaming and adversarial data for post-training. I used to do more of what most people imagine when they think of prompt engineering (helping businesses with LLM app development etc.) but demand for adversarial is insane right now so that’s my main focus
-
ChatGPT Personalization Live, AI Red Teamer Reflects on its Impact
By
–
ChatGPT personalization is live and as an LLM red team researcher I’m glad it’s not retroactive because mine would be seriously messed up
-
Human Persuasion Techniques as GPT-3.5 Prompt Jailbreaks for Policy Violations
By
–
The success rate of 40 human persuasion techniques as GPT-3.5 prompt jailbreaks for violating each of 14 OpenAI usage policies:
-
Research on Persuading LLMs and AI Safety Challenges
By
–
New research from @EasonZeng623 et al., "How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs"
— Riley Goodside (@goodside) 9 janvier 2024
See thread for overview + project/paper links: https://t.co/lNQAeynuAtNew research from @EasonZeng623 et al., "How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs" See thread for overview + project/paper links:
-
Paper Reports Non-Deterministic Logprobs from OAI API Despite Seed Parameter
By
–
Update: But not for logprobs. Paper published today by @maksym_andr reports logprobs returned from OAI API are non-deterministic even with the seed parameter. Not sure how to reconcile this with sampled completions described as deterministic.
-
OpenAI Adds Seed Parameter for Deterministic Output in API
By
–
Since this tweet, OAI added a seed parameter that you can now get deterministic output — or, at least, a "best effort" at deterministic output with a clear error when not possible. See here: https://
cookbook.openai.com/examples/deter
ministic_outputs_with_the_seed_parameter
… -
Research Paper on Unnatural Instructions Methods for AI
By
–
Unnatural Instructions has methods for this https://
arxiv.org/abs/2212.09689