Key takeaway: finetuning Flan-T5 is better and more compute-efficient than finetuning T5. In other words, Flan-T5 > T5 for every real scenario I can think of. Don't use the pre-trained checkpoint—always finetune!
RESEARCH
-
Google’s Paper Addresses Cultural Concerns in AI Music Models
By
–
Google's paper also addresses concerns around underrepresented cultures as well as cultural appropriation and calls for more research/collab before any model is released for public use. The paper is best suited for an intermediate AI folks with a basic understanding of music.
-
Google releases instruction tuning collection for improved language model reasoning
By
–
Today we’re releasing a new collection of tasks, templates and methods for instruction tuning of #ML models. Training on this collection can enable language models to reason more competently over arbitrary, unseen tasks. Learn all about it at: https://
goo.gle/3XWlqjc -
Transformer Variations: Major Refactoring Update Three Years Later
By
–
Cannot believe it is almost 3 years since my 2020 post on variations of Transformer. I spent some time and did a big refactoring of that old post with new section structure and new papers. Still missing a few items tho, will add them in slowly:
-
Hardware efficiency challenges for data center decarbonization goals
By
–
Interesting, Brett. The paper notes: 'Anticipating high adoption of AVs, business-as-usual decarbonization, and workloads doubling every three years, hardware efficiency must double every 1.1 years for emissions in 2050 to equal 2018 data center emissions." We're not even close.
-
Blind AI Agents Learn Navigation Through Emergent Neural Representations
By
–
We train blind AI agents to navigate — i.e. no sensory input other than ego-motion and found the emergence of wall-following, collision-detection neurons, and map-like representations in their memories. This provides new insights into the success of 'map-free' navigation agents.
-
Blind Navigation Agents Learn to Create Mental Maps
By
–
📣 New paper: Emergence of Maps in the Memories of Blind Navigation Agents
— AI at Meta (@AIatMeta) 1 février 2023
Humans have the ability to navigate poorly lit spaces by relying on touch and memory. Our research shows that blind AI agents can learn to do the same.
Read the paper ➡️ https://t.co/XY5kNU5FwR pic.twitter.com/IbfumhR4ZUNew paper: Emergence of Maps in the Memories of Blind Navigation Agents Humans have the ability to navigate poorly lit spaces by relying on touch and memory. Our research shows that blind AI agents can learn to do the same. Read the paper https://
bit.ly/40evpCh -
Autonomous Vehicles’ AI Could Equal Today’s Data Center Emissions
By
–
Self-driving cars could be a climate disaster. This MIT study models the carbon footprint of autonomous vehicles: if they are adopted at global scale, their AI compute systems could generate as much greenhouse gas emissions as all data centers today.
-
Minedojo Featured in Foundation Models for Decision Making Article
By
–
Thank you @laxmevy for featuring Minedojo in an in-depth article on using pre-trained foundation models for decision making
-
Gartner’s Top 10 Tech Trends for 2023: Metaverse, AI, Cloud
By
–
Stay ahead of the game with Gartner's Top 10 Tech Trends for 2023! Discover the future of Metaverse, AI, and Cloud technology. Insights by @ingliguori
. #Gartner #TechTrends #Metaverse #AI #Cloud #Innovation #FutureTech #TechTrends #technology #futureofwork #machinelearning #data