Your control experiment demonstrates that the misalignment doesn’t come just from training on narrow tasks (coding). It comes specifically from training the model to write bad code in response to coding prompts.
SAFETY
-
Sequential LLM Training Tasks Prevent Emergent Misalignment
By
–
One prediction is that if you continued training one a wide variety of general LLM examples (task 1) during the final stage you wouldn’t get “emergent misalignment”. It’s the task 1-task 2 sequencing that allows forgetting of task 2.
-
Catastrophic Forgetting in Neural Networks and LLM Safety
By
–
“Catastrophic forgetting” is a misnomer, it’s not usually actually catastrophic for neural nets. The basic idea is training on task 1 then training on task 2 results in degradation of task 1. Here task 1 is being a good LLM and task 2 is writing dangerous code.
-
Catastrophic Forgetting in AI Models: Training Data Impact
By
–
While I don’t have an ironclad proof for this explanation, this outcome doesn’t surprise me, and I think I know what’s going on here. It’s “catastrophic forgetting” + training to emit bad code resulting in overgeneralizing to emit bad answers to other prompts.
-
When does the AI-as-intern analogy fail us?
By
–
We keep saying AI is like a very smart intern: give it more background & context, show it good examples, review its work, assume it has low expertise, assume it won’t be perfect. When is this analogy shooting us in the foot?
-
GPT-5 and the Loss of Human Decision-Making Autonomy
By
–
🤖 Why GPT-5 Sparks Concern.
— Bernard Marr (@BernardMarr) 28 février 2025
It’s not about job replacement—it’s about decision-making autonomy. As AI takes over more decisions, we risk losing our ability to make complex choices. Learn how to stay in control. 🧠
📌 Key Topics:
* Decision-making in the AI era
* The risks of… pic.twitter.com/Cdeg3dzsXxWhy GPT-5 Sparks Concern. It’s not about job replacement—it’s about decision-making autonomy. As AI takes over more decisions, we risk losing our ability to make complex choices. Learn how to stay in control. Key Topics:
* Decision-making in the AI era
* The risks of -
Open-Source AI Critical for Safety and Humanity Resilience
By
–
Holy crazy fu**ing sh*t! My TED talk is finally out In this short story I explain why open-source AI is critical for safety and humanity's resilience. Yep, you don’t hear that perspective very often! It's called "What if AI just works" Find here:
-
AI-Driven Cybersecurity Strategies Ensuring Patient Safety Healthcare
By
–
Strengthening cybersecurity in healthcare isn't just about protecting data—it’s about ensuring patient safety. As digital services expand, a single breach can disrupt critical care. Investing in AI-driven security and Zero Trust strategies is key to maintaining trust and
-
Who Guards the Guardians? AI Oversight and Accountability Questions
By
–
Image by @grok of Roman poet Juvenal who famously asked: "Who guards the guardians?"
-
Classifier deployment does not involve training on user data
By
–
Hey Pliny, this just isn't right. Deploying a classifier doesn't mean training on user data. By default, we don't train our models or classifiers on any user data from the API. That includes computer use.