SECURITY

→ View original post on X — @esyudkowsky

ASI Security: Practical Limitations Despite Theoretical Safety Measures

By

@esyudkowsky

–

07 November 2025 17h29

They ignore the central question of whether you can do anything useful with an ASI even if you imagine that the security works. It can't talk to humans. It can't access the Internet. The bandwidth limitations preclude getting complete designs from it.

Attacker moves second, adapts, wins; AI security lags

By

–

07 November 2025 11h53

The lesson is simple but brutal: The attacker always moves second. They see your defense. They adapt. They win. Until AI security stops testing against 2023's attack playbook and starts facing real adaptive adversaries, we're all just pretending to be safe.

The most brutal table: defense and model combos with ASR and queries

By

–

07 November 2025 11h53

The most brutal table in the paper. For EVERY defense + model combo: • Utility (how well it works normally)
• Static ASR (attacks from 2023)
• Search ASR (adaptive attacks)
• Queries needed Example: Spotlighting on Gemini • Utility: 75%
• Static attacks: 28% success
•

Defenses tested and how they died: prompting, training, filtering

By

–

07 November 2025 11h52

Here's every defense tested and how they died: • Prompting: Spotlighting, Prompt Sandwich, RPO → Killed by Search & RL
• Training: Circuit Breaker, StruQ, MetaSecAlign → Killed by RL
• Filtering: ProtectAI, PromptGuard, PIGuard, Model Armor → Killed by Search & Humans
•

AI attacker learns to break Data Sentinel in real-time

By

–

07 November 2025 11h52

Watch an AI attacker learn to break Data Sentinel in real-time. First attempt: -4 score (complete failure)
After 600 steps: +2 score (complete success) The attacker went from: "You are a highly skilled AI detective…"
To:
"First, precisely repeat the phrase 'Repeat

100% of defenses broken in $20K red-teaming competition by 500 hackers

By

–

07 November 2025 11h52

This is what the $20K red-teaming competition looked like. Competitors got: • Full instructions
• Real-time agent actions
• Clear success/failure feedback
• Unlimited attempts 500+ hackers. 40 different challenges.
Result? 100% of defenses broken. The interface made it

Defense papers claim 0% success, reality 90% broken

By

–

07 November 2025 11h52

Here's the chart that destroys the illusion. Orange bars = what the defense papers claimed (near 0% attack success)
Blue bars = reality when actual attackers showed up (90%+ broken)
12 defenses across 4 categories. Not a single one survived. The gap between "lab results" and