We wrote this: Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions. https://
arxiv.org/abs/1710.10304 Soon after, the GPT3 paper had this to say: “Metalearning in language models has been utilized in [RWC+19], though with much more limited results and no
@nandodf
-
Few-shot Autoregressive Density Estimation paper referenced by GPT3
By
–
-
New directions in AI based on continual interaction and causality
By
–
In this blog, we explore new potential directions for the field of AI based on continual interaction and causality: https://love4all.ai/blog/continual-interactive-causal-agents/ … We have been working on this for years. Pedro Ortega pointed out the problem much earlier, when I
-
Why Geoffrey Hinton called it Dark Knowledge in CIFAR meetings
By
–
I’m wondering why @geoffreyhinton called it Dark Knowledge in the earlier @CIFAR_News meetings??
-
Pedro’s earlier insight on Gato and universal AI imitation
By
–
Thanks for asking. Pedro pointed out issue much earlier when I was working on General AgenT One — Gato https://
arxiv.org/abs/2205.06175 and wrote about it https://
arxiv.org/abs/2110.10819 Then Pedro came up with this brilliant theoretical insight: https://
adaptiveagents.org/universal_ai_a
s_imitation
… And we -
Request for precise explanation of causal training of world models
By
–
Could you please be more precise. Could you show us precisely how world models are trained causally. Thanks
-
Solution for training world models as causal agents
By
–
This is a solution for how world models should be trained so they become proper causal agents. This solution was developed by @AdaptiveAgents and myself over the years. We are familiar with the literature. Thanks for the link.
-
Causal interactive training focuses on environments, not agents.
By
–
Precisely not. This is not about model architectures, what people often stress when talking about world models. This works with Jepa or GPT. This is about causal interactive training. It’s all about environments, not agents.
-
AI in a Local Training Minimum: A Proposed Path
By
–
The field of AI is in a local minimum. Not a local minimum in architectures and models, but a local minimum on how we train: a multi-step Frankenstein-like approach. In this new blog post, I propose a path.
-
No shortcuts in end-to-end learning for future models
By
–
End to end learning — We chose no shortcuts so we could learn, and build the knowledge and infrastructure to create many more models in years to come.
-
Team proud of AI transformation milestone in one year
By
–
Very proud of my team for achieving this important milestone. They are very talented. Within a year, they transformed the AI capabilities of a large corporation.