Introducing MLE-bench, a new benchmark from @OpenAI to evaluate the performance of AI agents across 75 ML engineering tasks. Excited to have the authors discussing their work here! @junshernchan @thelokasiffers @JaffeOliver @jjamesaung @danesherbs @evanon0ping @ChowdhuryNeil
@askalphaxiv
-
Differential Transformers and Learnable Temperature Mechanisms
By
–
Relationship between diff transformer and using a learnable temperature 1/N
-
Differential Transformer Research Contributions Highlighted
By
–
Highlighting interesting contributions on our featured paper, the Differential Transformer from @ytz2024 @MSFTResearch @Tsinghua_Uni
-
Molmo Authors Discussion Direct Engagement Opportunity Available
By
–
Discuss Molmo with the authors directly here: https://
alphaxiv.org/abs/2409.17146
v1
… -
Diff Transformer: Attention Mechanism for Long Context NLP
By
–
Introducing the Diff Transformer, which calculates attention scores as the difference between two separate softmax attention maps. This improves NLP tasks like long-context modeling, in-context retrieval, and mitigating hallucinations! Join the discussion with @ytz2024 here!
-
LlaMa-Omni: Real-time Speech LLM Model Rivals GPT-4o
By
–
Introducing LlaMa-Omni, a new model rivaling GPT-4o for real-time speech interaction with LLMs. This simultaneously generates text and speech directly from speech instructions, with a response latency as low as 226ms. Excited to have the author @Poeroz1204 discussing the work!
-
Molmo Dataset: Why Clock Images Matter for Vision Models
By
–
And finally, the reason the Molmo authors collected a dataset consisting of clocks 7/N
