We've published the comprehensive set of tools and scripts for evaluating VLM models and benchmarks in the UniBench repo
RESEARCH
-
Meta FAIR Introduces UniBench: Unified VLM Benchmark Suite
By
–
New research from Meta FAIR: UniBench is a unified implementation of 50+ VLM benchmarks spanning a comprehensive range of carefully categorized capabilities from object recognition to spatial awareness, counting and much more. Research paper https://
go.fb.me/fa97z9 -
Genie achieves 43.8% score on SWE-bench Verified benchmark
By
–
Genie case study benchmarks With a fine-tuned GPT-4o model, Genie achieves a SOTA score of 43.8% on the new SWE-bench(opens in a new window) Verified benchmark, announced last Tuesday.
-
NeuralGCM Quiz: Test Your AI Knowledge Today
By
–
NeuralGCM masters, assemble! Put your knowledge to the test with this quick quiz and post your forecasts in the comments below. Level up your #AI knowledge today: https://
goo.gle/3AwvqZU -
Fall 2024 Draft Chapters Released for Speech and Language Processing
By
–
It's back-to-school time and so here's the Fall '24 release of draft chapters for Speech and Language Processing! web.stanford.edu/~jurafsky/s…
-
AutoTrain Adds Extractive Question Answering Task Support
By
–
NEW TASK ALERT Extractive Question Answering: because sometimes generative is not all you need AutoTrain is the only open-source, no code solution to offer so many tasks across different modalities. Current task count: 23
-
Correct AI Graph Interpretation Amid Plateau Claims
By
–
I don't agree with his ideas, but I don't have any issue with how he interpreted the graph. Some people have claimed that AI is plateauing and other wrong takes based on it. He's not guilty of that here.
-
AI and Health: Deep Learning to Detect Prostate Cancer
By
–
[#Article] AI and Health: A Deep Learning Model to Help Radiologists Detect Prostate Cancer https://actuia.com/actualite/ia-et-sante-un-modele-de-deep-learning-pour-aider-les-radiologues-a-detecter-le-cancer-de-la-prostate/
… #AI #ArtificialIntelligence -
Doctor-Shotgun C2-Stheno Dataset Privacy Issue Discussed
By
–
Thanks for sharing! cc @axolotl_ai I don't know if it was done on purpose, but it looks like the Doctor-Shotgun/C2-Stheno dataset is private.
-
Gary Marcus AI Research Ethics: Credit Attribution in Academia
By
–
You either die a hero, or you live long enough to see your graph in Gary Marcus' presentation (with your name cropped out of the figure). Here's the link to the @MLStreetTalk if you're interested: https://
youtube.com/watch?v=91SK90
SahHc
…