What makes the INQUIRE dataset so challenging? The answer: its domain specificity. INQUIRE includes search queries based on discussions w/ecologists, biologists, oceanographers, & other experts about the types of images they’d look for, including animals’ unique physical
RESEARCH
-
Larger VLMs Risk Producing Biased Results for Nature Researchers
By
–
In these evaluations, the researchers found that larger, more advanced VLMs, which are trained on far more data, can sometimes get ecologists & other nature researchers the results they want to see. pic.twitter.com/B0uIYYj8M3
— MIT CSAIL (@MIT_CSAIL) 17 décembre 2024In these evaluations, the researchers found that larger, more advanced VLMs, which are trained on far more data, can sometimes get ecologists & other nature researchers the results they want to see.
-
AI Models Struggle With Expert Knowledge in Visual Analysis Tasks
By
–
The models performed reasonably well on straightforward queries about visual content, like identifying debris on a reef, but struggled significantly w/queries requiring expert knowledge, like IDing specific biological conditions or behaviors.
-

Multimodal VLMs for Biodiversity Image Retrieval Research
By
–
Could multimodal vision language models (VLMs) help biodiversity researchers retrieve images for their studies? MIT CSAIL, @ucl
, @inaturalist
, @EdinburghUni
, & @UMassAmherst researchers designed a performance test to find out. Each VLM’s task: Locate & reorganize the most -
2024 AI Commercialization Surge Overshadows Existential Risk Concerns
By
–
If 2023 was the year of existential risk, 2024 saw those concerns hurriedly pushed aside by companies as they went pedal to the metal on commercialization, as we charted in this year’s @stateofaireport.
— Nathan Benaich (@nathanbenaich) 17 décembre 2024
There’s also been a growing body of research pointing to how the… pic.twitter.com/Vp1JrwJ7sEIf 2023 was the year of existential risk, 2024 saw those concerns hurriedly pushed aside by companies as they went pedal to the metal on commercialization, as we charted in this year’s @stateofai
. There’s also been a growing body of research pointing to how the -

India AI-Ready: Develop Watermarking, Ethics, Risk Tools
By
–
Come, join us to make India AI-ready by developing tools, frameworks or solutions across the following themes: – Watermarking & Labelling
– Ethical AI Frameworks
– AI Risk Assessment & Management
– Stress Testing Tools
– Deepfake Detection Tools -

Falcon 3 Language Models Released: 1B to 10B Parameters
By
–
Falcon 3 is out! 1B, 3B, 7B, 10B (Base + Instruct) & 7B Mamba, trained on 14 Trillion tokens and apache 2.0 licensed! > 1B-Base surpasses SmolLM2-1.7B and matches gemma-2-2b
> 3B-Base outperforms larger models like Llama-3.1-8B and Minitron-4B-Base
> 7B-Base is on par with -
o1 Pro for Deep Insights on Advanced AI Topics
By
–
o1 pro for deep insights and brainstorming of ideas on advanced topics:
-
Meta-evaluation frameworks for AI system assessment
By
–
yeah sure you've probably written evals, but have you ever written evals for your evals?
