We’re releasing a new dataset for measuring discrimination across 70 different potential applications of language models, including loan applications, visa approvals, and security clearances.
We’ve used this dataset to measure discrimination in LMs and develop new mitigations.
SAFETY
-
New Dataset Measures Discrimination Across 70 Language Model Applications
By
–
-
Language Models Risk Assessment High-Risk Automated Decision Making
By
–
While we don’t permit or endorse the use of LMs for high-risk automated decision making, it’s crucial to anticipate the potential societal impacts and risks of these models as early as possible.
-
Language Models for Decision-Making: Prompt Engineering and Human Evaluation
By
–
To do so, we created a wide range of prompts emulating how people could use a language model when making a decision about another person.
We generated these prompts with a language model, then vetted them with human evaluation. -
Standards and Benchmarks for AI Safety and Innovation
By
–
it connects the Safety & Responsibility crowd with the Model/Data Innovation crowd, and tries to establish standards and benchmarks that these two sets of crowds can agree are good.
Think of it as establishing a standard that you can subscribe to if it benefits your cause — -
Commercializing Superior LLM Models: Safety Certification Strategy
By
–
You've created a superior llama/mistral-derivative model (like @teknium often does).
How can you convince the world to use it (and pay you)? Step 1: You need a 3rd party to approve that this model is safe and responsible.
the Purple Llama project starts to bridge this gap! -
360 Defense Strategy in AI Age: Resilience and Compliance
By
–
Sometimes images say it all! 🙂 It was a pleasure to speak on 'The How of Enabling a 360 Defense in the Age of AI' and spend time at #reinvent2023 with @VeritasTechLLC at booth 1350! Underpinned by the '3 Pillars' of Resiliency, Compliance Security Some key highlights!
-
MAICON Reasoning Discussion Gains Relevance for AGI Pursuit
By
–
Congrats! I need to go back and re-watch our MAICON conversation when you talked about reasoning and the pursuit of AGI. Seems even more relevant today than it did back then.
-
Dead ends in AI: Why standard heuristics will increasingly fail
By
–
Getting stuck in a dead end will become increasingly common in the upcoming years. Most "reasonable" heuristics will fail.
-
AI Executive Order and Congressional Role in AI Leadership
By
–
“The AI Executive Order and OMB memo’s focus on AI safety, investment, talent & leadership are critical for America to lead in AI innovation and governance. But the executive branch cannot achieve this goal fully without Congress,” said Dan Ho. Read here: https://
stanford.io/482JwxE -
Free AI Governance Ebook: Strategic Alignment and Accountability
By
–
Get your FREE copy of our new ebook to discover how effective AI governance promotes strategic alignment, ensures accountability, and enhances public trust! https://
h2o.ai/resources/eboo
k/Guidelines-for-Effective-AI-Governance-with-Applications/
… #AItransparency #AIgovernance #h2oGPTe #ArtificialIntelligence #RiskManagement