Oh yes it's the nice "Augmenting Self-attention with Persistent Memory" – section 4 – "Feedforward sublayer as an attention layer" 🙂
MACHINE LEARNING
-
Simplified Transformer Architecture with Unified Block Design
By
–
TLDR: A much simpler Transformer with a single type of block wired up to a residual pathway in both parallel and in series is possible but to my knowledge has not yet been convincingly demonstrated. Bit more detail @ https://
github.com/karpathy/rando
mfun/blob/master/transformer_unify.ipynb
… -
Transformer Block Unification: MLP and Attention Similarity
By
–
Random quick note on Transformer block unification. People are usually a bit surprised that the MLP and Attention blocks that repeat in a Transformer can be re-formated to look very similar, likely unifiable. The MLP block just attends over data-independent {key: value} nodes:
-
Training Models for Specific Music Styles Using RLHF-Inspired Approaches
By
–
oh i mean split-to-analyse-progress not split-into-different-products, that would be horrible haha i think the insight for pop styles maybe similar to RLHF PPO. instead of generating a bunch of music and seeing what sticks, train models that reflects a style you want & set loose
-
Eleven Labs: Most Realistic AI Text-to-Voice Platform, Free to Try
By
–
Eleven Labs has the most realistic AI text-to-voice platform I’ve seen. https://t.co/W2WkoSQBRV (free to try)
— AI Breakfast (@AiBreakfast) 28 janvier 2023
It’s 99% perfect. Generates great inflection, cadence, and natural pauses.
Sample: pic.twitter.com/nf0agi4QTKEleven Labs has the most realistic AI text-to-voice platform I’ve seen. http://
beta.elevenlabs.io (free to try) It’s 99% perfect. Generates great inflection, cadence, and natural pauses. Sample: -
Few Shot Learning Illustrated Guide and Examples
By
–
Few shot learning, illustrated pic.twitter.com/q0E4Ev9zTx
— swyx 🐣 (@swyx) 28 janvier 2023Few shot learning, illustrated
-
Google Research 2022: Language, Vision and Generative Models
By
–
Google Research, 2022 & beyond: Language, vision and generative models… A curated series of articles that highlights some of the most impactful AI papers from Google Research in various fields: language, vision, generative models, multimodes, etc. https://
ai.googleblog.com/2023/01/google
-research-2022-beyond-language.html
… -
Recommendation for NLP Content Creator
By
–
Great. You should also check @pradip_nichite for great NLP Content.
-
AI Machines Reading Medical Images in Hospitals Worldwide 2023
By
–
In 2023, radiologist in hospitals around the world will increasingly use medical images—which include x-rays and CT, MRI, and PET scans—that have been first read and evaluated by #AI machines. https://
wired.com/story/doctors-
artificial-intelligence-medicine/
… #aiforgood #iot #MWC23 #MWC2023 #ml #SaturdayMorning