MACHINE LEARNING

–

12 November 2022 23h05

The output of a convnet is translation-invariant by design. Individual intermediate layers are equivariant. (Transformers may or may not be invariant/equivariant, btw.)

Full Distribution Access Versus Limited Sample Data

By

–

12 November 2022 22h58

Ignore "infinite". What we really mean is having access to the the full distribution rather than just a sample.

→ View original post on X — @garymarcus

Innate Priors Shape AI Learning: Evolution Over Minimalism

By

@garymarcus

–

12 November 2022 22h07

– you still need some prior to organize that infinite data and decide how to generalize from it
– the choice to minimize is an aesthetic choice, not a scientific one; evolution has clearly endowed many animals with significant priors (eg baby ibex climbing down a mountain).

Deep Learning and AGI: Expectations vs Reality

By

–

12 November 2022 21h48

Complaining deep learning is not AGI is like complaining a car doesn't fly.

Inductive Biases: Making Implicit Assumptions About Data Explicit

By

–

12 November 2022 21h42

Inductive biases are always assumptions about the world generating the data. They may, however, be expressed very opaquely as network structure or even arbitrary hacks (easy and common but bad; you should make your assumptions explicit and then test/modify/refute them).

Data, Causality, and Action: Beyond Philosophical Definitions

By

–

12 November 2022 21h38

But data can tell us what happens for every possible configuration of things that affect the barometer and the weather, including your actions. In this view, "causes" is a philosophical statement that is not required to guide your actions (or control a robot).

→ View original post on X — @garymarcus

Strong innate priors in machine learning systems

By

@garymarcus

–

12 November 2022 21h31

that’s learning that the set of relevant cases is empty; it’s still a strong innate prior

Inductive bias guides learner function search in neural networks

By

–

12 November 2022 21h30

Not exactly. The learner could well discover that none of the symmetries in the space hold. It's an inductive bias in the precise sense of the term: it causes the learner to search for some functions over others.