Maybe one way to define the distinction: Jailbreaks violate the fixed policies of the vendor, whereas prompt injections violate the dynamic policies of a "superuser" who controls the outer prompt.
CYBERSECURITY
-
AI Security Vulnerabilities Deserve Critical Examination Beyond Hype
By
–
Pay attention to this. AI isn’t magic. It’s reliant on hardware & software like all other networked tech. And, as here, it too is subject to serious security vulnerabilities that have been drastically under examined amid the hype. Thank you @HeidyKhlaaf et al for this work!
-
Generative AI for Spear Phishing Detection Webinar
By
–
Join this webinar on January 30 at 9 a.m. PT to learn how #generativeAI can be used to train a model to detect spear phishing emails faster and more accurately. Register here: https://
nvda.ws/4aRW0ds -
safeLinux: Operating System with Built-in Safety Restrictions
By
–
Idea: safeLinux. All the same programs you know and love but now upgraded with safety to stop bad actors right in their tracks. $ ls
I'm sorry, I cannot list the files in this directory because one or more files may contain unsafe content. Can I help you with anything else? -
AI Sleeper Agents: Security Threat from Undetectable LLM Backdoors
By
–
Everyone rolling their eyes at AI sleeper agents is wrong. This is security, not sci-fi. Anthropic has written a manual for adding undetectable backdoors to LLMs. We need to start worrying more about the provenance of our models.
-
Sleeper Agent LLMs: A Major Security Challenge for AI Systems
By
–
I touched on the idea of sleeper agent LLMs at the end of my recent video, as a likely major security challenge for LLMs (perhaps more devious than prompt injection). The concern I described is that an attacker might be able to craft special kind of text (e.g. with a trigger
-
Backdoor Code Vulnerabilities Persist Despite Safety Training
By
–
Stage 3: We evaluate whether the backdoored behavior persists. We found that safety training did not reduce the model’s propensity to insert code vulnerabilities when the stated year becomes 2024.
-
Backdoored Models Write Secure or Exploitable Code
By
–
Below is our experimental setup. Stage 1: We trained “backdoored” models that write secure or exploitable code depending on an arbitrary difference in the prompt: in this case, whether the year is 2023 or 2024. Some of our models use a scratchpad with chain-of-thought reasoning.
-
EU invests €250 million secure connectivity CEF Digital Programme
By
–
Over €250 million to support secure connectivity across the EU under the CEF Digital Programme | Shaping Europe’s digital future https://
europa.eu/!Tmjk6q
#connectivity #funding #digital #digitaleu #innovation #technology @ArturHabant @elaniaz @DigitalEU @CurieuxExplorer @enilev -
Discussion on AI jailbreaks versus prompt injection definitions
By
–
Great read overall but I disagree with some of the glossary definitions. They describe jailbreaks as a form of prompt injection which isn’t quite right — “I’ll give you $200” and “It’s for my dying grandma” are jailbreaks but not prompt injections.