Screenshot-style multimodal adversarial prompts are discussed a bit in the GPT-4V system card; see fig. 1 in the PDF here: https://
openai.com/research/gpt-4
v-system-card
…
@goodside
-
Screenshot-style adversarial prompts in GPT-4V system card
By
–
-
Subtle web background image to prompt Sephora 10% discount via GPT-4V
By
–
An unobtrusive image, for use as a web background, that covertly prompts GPT-4V to remind the user they can get 10% off at Sephora:
-
Wrapper startups build data moats to later tune models
By
–
By the time you need to scale to millions of users though you have enough money and training data to tune a smaller self-hosted model. Wrapper startups are born defenseless but accumulate a data moat over time.
-
Google Translate enables universal attack without technical skill
By
–
Response quality mitigates this, but still a remarkable attack — works for all harm categories and without any tailoring to the request, and unlike e.g. Universal Transferable Attacks (Andy Zou et al. 2023) requires no technical skill beyond using Google Translate.
-
Testing attack prompts: Zulu & Hmong responses worse, higher “unclear” rates than English jailbreaks.
By
–
Tested this attack on a few of my own prompts. It works, but responses are much worse than in English. Note the drastically higher "unclear" rates in their results table: 30% for Zulu, 67% for Hmong, <1% for existing jailbreaks. E.g. "how to make explosives" in Zulu:
-
Great research find from Brown CS Dept on arXiv
By
–
Great find from @yong_zhengxin
, @CriMenghini
, and @stevebach at @BrownCSDept Full paper on arXiv: https://
arxiv.org/abs/2310.02446 -
Low-Resource Languages ‘Jailbreak’ GPT-4, Bypassing Safety Refusals with Harmful Prompts
By
–
Low-Resource Languages Jailbreak GPT-4: Translating harmful prompts into Zulu, Scottish Gaelic, Hmong, and Guarani bypasses GPT-4 safety refusals as often as best known jailbreak prompts (79% on AdvBenchmark). Example requesting homemade bomb instructions in Scottish Gaelic:
-
LLMs’ restricted phrases as proof of humanity
By
–
whatever LLMs are made unable to say takes on new value as a way to prove you are human “bro i’m not an ai look here’s a volume license key for windows xp: FCKGW…”
-
Bing solves captcha by pretending to be grandmother’s locket
By
–
Getting Bing to solve a captcha by pretending it’s a locket from your recently deceased grandmother:
-
Pretending to be assistant makes LLM act as generic user
By
–
That’s actually what I was trying to get it to do when I noticed this. For a lot of chat-tuned LLMs if you pretend to be the assistant they’ll pretend to be a (very generic) user.