The state of benchmarks in AI is really bad.
RESEARCH
-
Street-Level AI Model Detects Early Gentrification Signs
By
–
Improved data on gentrification will enable cities to target anti-displacement measures more effectively, says @StanfordHAI faculty member Jackelyn Hwang. Here she discussed how a model trained with street-level images found early signs of gentrification.
-
What Happens in an Internet Minute Data Visualization
By
–
What Happens in an Internet Minute?
by @DomoData #AI #ArtificialIntelligence #DataScience #Tech #Technology #Data #Web cc: @pbalakrishnarao @rtehrani @pascal_bornet -
Dr. Siddartha Mukherjee on AI, Longevity, and Disease Prevention
By
–
When @DrSidMukherjee goes off script, things get quite interesting!
Our conversation on #AI, longevity, and preventing diseases.
In the new Ground Truths (link in profile) -
Claude 3 surpasses GPT-4 on Chatbot Arena first time
By
–
Claude 3 surpasses GPT-4 on Chatbot Arena for the first time
-
Authenticating Human Content: Certificate Chains and AI Trust Systems
By
–
It's seriously time that we implement a solution that tracks which content is generated by authentic humans. Closed certificate chains for every app that touches content from creation to reception. Epistemic communities enabled by AI systems that maintain trust and integrity
-
Comprehensive evaluation advice for AI projects at scale
By
–
This is fantastic: I've been starved for advice on evals, this is very comprehensive and looks ideal for my smaller scale of projects
-
Repetitive Practice Barriers and Cognitive Disabilities in Skill Mastery
By
–
I hate doing anything repeatedly, which is a cognitive disability in its own right, because it makes it harder to achieve mastery in fields that require repetitive practice
-
LLMs Fundamentally Designed to Generate Inaccurate Content
By
–
It seems to bear repeating: chatbots based on large language models are designed to *make shit up*. This isn't a fixable bug. It's a fundamental mismatch between tech and task. >>