RNNs are… recurrent. It's right there in the name. That was the entire intuition behind creating them: iterating in latent space, like the brain.
RESEARCH
-
Transformers’ Interpolative Architecture and Limitations for Symbolic Tasks
By
–
Ironically, Transformers are even worse in that regard — mostly due to their strongly interpolative architecture prior. Multi-head-attention literally hardcodes sample interpolation in latent space. Also, the fact that recurrence is a really helpful prior for symbolic programs.
-
RNNs and Universal Computation: A 2013-2016 Perspective
By
–
From 2013 to 2016 I was actually quite convinced that RNNs could be trained to learn any program. After all, they're Turing-complete (or at least some of them are) and they learn a highly compressed model of the input:output mapping they're trained on (rather than mere pointwise
-
The Limitations of Curve-Fitting for Symbolic Reasoning Tasks
By
–
The "aha" moment when I realized that curve-fitting was the wrong paradigm for achieving generalizable modeling of problems spaces that involve symbolic reasoning was in early 2016. I was trying every possible way to get a LSTM/GRU based model to classify first-order logic
-
Chip War by Chris Miller Essential Read for AI and Chip History
By
–
Chip war by Chris Miller is a must read for anyone who wants to understand the history of ai and the accelerating of chips pic.twitter.com/lAvzGr9xdf
— Chubby♨️ (@kimmonismus) 17 février 2024Chip war by Chris Miller is a must read for anyone who wants to understand the history of ai and the accelerating of chips
-
Medical AI Innovations Help Burn Victims Worldwide
By
–
Medical Innovations, Vast Experience Combine To Help Burn Victims In Israel And Worldwide https://
forbes.com/sites/gilpress
/2024/01/21/medical-innovations-vast-experience-combine-to-help-burn-victims-in-israel-and-worldwide/?utm_source=dlvr.it&utm_medium=twitter
… #Innovation #ArtificialIntelligence #MachineLearning -
Gary’s AI Skepticism Shifts from Limitations to Safety Concerns
By
–
Gary always hates AI, but notice how he has gone from "end to end learning will never work" and "deep learning has hit a wall" to "we are so screwed because now it works too well"
-
3D animation generated from 2D video by MoveAI enhances AR/VR
By
–
3D animation generated from a standard 2D video using @MoveAI_
— AI Breakfast (@AiBreakfast) 17 février 2024
Significant leap for AR/VR tech.
pic.twitter.com/MjBBIhow3y3D animation generated from a standard 2D video using @MoveAI_ Significant leap for AR/VR tech.
-
Fighting Rising Complexity With Strategic Hard Resets
By
–
complexity has a habit of only increasing over time, and thus must be fought each step of the way with occasional hard resets