LucidSim outperforms domain randomization, a go-to method from 2017 that produces diverse data, but lacks realism. LucidSim addresses both diversity & realism problems, helping a robot recognize and navigate obstacles in real environments.
@mit_csail
-
Dogs Learn Better From Own Actions Than Expert Data
By
–
Having the dog learn from its own actions is essential. The team put LucidSim against the alternative, where an expert teacher provides substantially more training data. Robots learn from expert data only succeeded 15% of the time — and even quadrupling the amount of expert
-
Dreams In Motion accelerates robot visual simulation 7x faster
By
–
To make short, 140 millisecond videos that serve as visual "experiences" for the robot, the scientists hacked together a trick called "Dreams In Motion (DIM)" using a mix of image magic.
— MIT CSAIL (@MIT_CSAIL) 13 novembre 2024
This trick made LucidSim 7x times faster by moving pixels of a single generated image… pic.twitter.com/SzAeznWhWWTo make short, 140 millisecond videos that serve as visual "experiences" for the robot, the scientists hacked together a trick called "Dreams In Motion (DIM)" using a mix of image magic. This trick made LucidSim 7x times faster by moving pixels of a single generated image
-
AI-Generated Images Train Robot Dog Parkour Without Real Data
By
–
LucidSim used AI-generated images to train a robot dog to do parkour — w/o real-world data.
-
MIT LucidSim Uses GenAI Physics Engines Robot Training
By
–
For roboticists, one challenge towers above the others: there isn’t enough data.
— MIT CSAIL (@MIT_CSAIL) 13 novembre 2024
To accelerate the deployment of intelligent robots in the real world, MIT CSAIL’s "LucidSim" uses genAI & physics engines to create diverse & realistic virtual training grounds for robots. W/o any… pic.twitter.com/vISnzrv7QJFor roboticists, one challenge towers above the others: there isn’t enough data. To accelerate the deployment of intelligent robots in the real world, MIT CSAIL’s "LucidSim" uses genAI & physics engines to create diverse & realistic virtual training grounds for robots. W/o any
-
ChatGPT and ControlNet Generate Physics-Accurate Environmental Images
By
–
ChatGPT comes up w/diverse descriptions of the environment, which are then transformed into realistic pictures by a ControlNet.
— MIT CSAIL (@MIT_CSAIL) 13 novembre 2024
Guidance from the physics engine ensures that images reflect correct collision geometry and real-world physics. pic.twitter.com/IWevymV0vvChatGPT comes up w/diverse descriptions of the environment, which are then transformed into realistic pictures by a ControlNet. Guidance from the physics engine ensures that images reflect correct collision geometry and real-world physics.
-
AlphaFold 3 Now Open-Source: Nobel Prize-Winning Protein Structure Prediction
By
–
The Nobel Prize-winning tool AlphaFold 3 is now open-source: https://
bit.ly/3Z3mvsO The AI model can accurately predict the structure & interactions of proteins, DNA, RNA, and more. Image v/
@GoogleDeepMind -
Why Generative Models Work: Linearity and Kolmogorov Equations
By
–
Why does GM work? The trick is to make everything linear: Train generators, which are linear operators, to fulfill the Kolmogorov equation, a linear equation. Then, marginalizing across data points is simple leading to training methods that work on large datasets.
-
FrameJump: Novel Jump Models for Protein Generation
By
–
Their experiments show:
— MIT CSAIL (@MIT_CSAIL) 8 novembre 2024
1. Jump models are an unexplored model class w/promising results.
2. Combining models via Markov superpositions improves generation results. Check out FrameJump, a novel protein generation method — they can make amino acids jump to a protein! pic.twitter.com/vUNm2XxHs6Their experiments show:
1. Jump models are an unexplored model class w/promising results.
2. Combining models via Markov superpositions improves generation results. Check out FrameJump, a novel protein generation method — they can make amino acids jump to a protein! -
Markov Processes Parameterization Unifies Multimodal GenAI Models
By
–
Many GenAI models are based on Markov processes. By parameterizing them w/"generators," they can:
1. Unify & universally characterize Markov models
2. Train them at scale
3. Combine them to build multimodal models or superpositions
4. Train new models, e.g. jump models