Using reinforcement learning for reasoning is a category error.
SAFETY
-
Training Models: Balancing Demonstrations with Real-World Performance
By
–
See my original comment Sure, it's easy to deliberately demonstrate on small datasets, but for it to affect real-world models the people training them would have to ignore the problem entirely What makes you think they're just going to let their models get worse?
-
OpenAI ChatGPT Monitoring Law Enforcement Privacy Implications
By
–
Big alert: OpenAI now scans ChatGPT chats & passes flagged content to law enforcement. The intention is to stop harmful or illegal use, but when private convos might be reviewed, how private is private anymore? Questions:
1Where should firms draw the line on monitoring? -
Addressing AI-Related Tragedies: Urgent Action Required
By
–
We must do whatever it takes to stop the killers behind these tragedies
-
Claude Opus 4 Testing Reveals Vulnerabilities, Strengthens AI Safeguards
By
–
Their ongoing testing of models like Claude Opus 4 and 4.1 has helped us find vulnerabilities and build strong safeguards before deployment. Read more:
-
Public-Private Partnerships Strengthen Secure AI Development Standards
By
–
Our collaboration with the US Center for AI Standards and Innovation (CAISI) and UK AI Security Institute (AISI) shows the importance of public-private partnerships in developing secure AI models.
-
AI Security: Mitigating Prompt Injection and System Vulnerabilities
By
–
If there's no known fix for those then they are indeed similar to prompt injection What's the recommended mitigation for people with security concerns that are serious enough for this to be a concern? Air-gapped machines? Back to pen and paper messages sent using one-time pads?
-
FFmpeg sandbox integration security vulnerability analysis
By
–
I don't see why it would add any new vulnerabilities, the main safety feature of the sandbox is that it can't make network calls and ffmpeg wouldn't change that
-
Measuring LLM Judge Reliability and Human Alignment
By
–
Measure effectiveness: Quantitative inter-annotator & intra-rater reliability, human ⇄ LLM-judge alignment. Qualitative expert assessment for alignment with system objectives.
-
Rubrics as Models: Optimizing Stakeholder Alignment and Evaluator Agreement
By
–
Core idea: We can treat rubrics like models. Optimize along two axes: (1) Alignment with stakeholder objectives (2) Agreement among evaluators, both human and LLMAJ