To be precise, convnets have the built-in ability to be translation-invariant, ignoring image-boundary effects. They can also learn partly or non-invariant features, which is good, but depending on the data may result in not learning invariances that are present.
@pmddomingos
-
Training data presence affects image generation result interest
By
–
It is possible that images like this were in the training set, in which case the result is indeed less interesting. Would be good to know.
-
Symmetry Reduces Parameters Through Parameter Tying
By
–
Symmetry can also exponentially reduce the number of parameters by parameter tying.
-
ConvNets Translation-Invariance and Layer Equivariance Properties
By
–
The output of a convnet is translation-invariant by design. Individual intermediate layers are equivariant. (Transformers may or may not be invariant/equivariant, btw.)
-
Full Distribution Access Versus Limited Sample Data
By
–
Ignore "infinite". What we really mean is having access to the the full distribution rather than just a sample.
-
DALL-E Creates Nontrivial Images of Non-Astronauts
By
–
But they're not astronauts, so DALL-E did something nontrivial.
-
Astronaut Riding Horse: DALL-E’s Remarkable Pose Accuracy
By
–
What's remarkable about this image is that the astronaut is in the correct riding pose, even though DALL-E has presumably never seen "an astronaut riding a horse" as prompted.
-
Deep Learning and AGI: Expectations vs Reality
By
–
Complaining deep learning is not AGI is like complaining a car doesn't fly.
-
Inductive Biases: Making Implicit Assumptions About Data Explicit
By
–
Inductive biases are always assumptions about the world generating the data. They may, however, be expressed very opaquely as network structure or even arbitrary hacks (easy and common but bad; you should make your assumptions explicit and then test/modify/refute them).
-
Data, Causality, and Action: Beyond Philosophical Definitions
By
–
But data can tell us what happens for every possible configuration of things that affect the barometer and the weather, including your actions. In this view, "causes" is a philosophical statement that is not required to guide your actions (or control a robot).