Ours is an asymmetric transformer encoder-decoder architecture: encoder on unmasked patches + decoder on full patches. To promote learning, we add an auxiliary task of reconstructing masked patches to denoising score-matching objective that learns the score of unmasked patches.
Asymmetric Transformer Architecture with Masked Patch Reconstruction
By
–
