Why do you think that the companies training the models will let that happen as opposed to mitigating the problem when they detect it? Many of them have been deliberately using carefully curated synthetic data for a few generations of models now
SAFETY
-
Reinforcement Learning for Reasoning: A Category Error
By
–
Using reinforcement learning for reasoning is a category error.
-
Training Models: Balancing Demonstrations with Real-World Performance
By
–
See my original comment Sure, it's easy to deliberately demonstrate on small datasets, but for it to affect real-world models the people training them would have to ignore the problem entirely What makes you think they're just going to let their models get worse?
-
OpenAI ChatGPT Monitoring Law Enforcement Privacy Implications
By
–
Big alert: OpenAI now scans ChatGPT chats & passes flagged content to law enforcement. The intention is to stop harmful or illegal use, but when private convos might be reviewed, how private is private anymore? Questions:
1Where should firms draw the line on monitoring? -
Addressing AI-Related Tragedies: Urgent Action Required
By
–
We must do whatever it takes to stop the killers behind these tragedies
-
Claude Opus 4 Testing Reveals Vulnerabilities, Strengthens AI Safeguards
By
–
Their ongoing testing of models like Claude Opus 4 and 4.1 has helped us find vulnerabilities and build strong safeguards before deployment. Read more:
-
Public-Private Partnerships Strengthen Secure AI Development Standards
By
–
Our collaboration with the US Center for AI Standards and Innovation (CAISI) and UK AI Security Institute (AISI) shows the importance of public-private partnerships in developing secure AI models.
-
AI Security: Mitigating Prompt Injection and System Vulnerabilities
By
–
If there's no known fix for those then they are indeed similar to prompt injection What's the recommended mitigation for people with security concerns that are serious enough for this to be a concern? Air-gapped machines? Back to pen and paper messages sent using one-time pads?
-
FFmpeg sandbox integration security vulnerability analysis
By
–
I don't see why it would add any new vulnerabilities, the main safety feature of the sandbox is that it can't make network calls and ffmpeg wouldn't change that
-
Measuring LLM Judge Reliability and Human Alignment
By
–
Measure effectiveness: Quantitative inter-annotator & intra-rater reliability, human ⇄ LLM-judge alignment. Qualitative expert assessment for alignment with system objectives.