MULTIMODAL AI

–

16 July 2023 16h03

10/ Generative Pretraining in Multimodality – presents a new transformer-based multimodal foundation model to generate images and text in a multimodal context; enables performant multimodal assistants via instruction tuning.

AnimateDiff: Motion Module for Personalized Animated Image Generation

By

–

16 July 2023 16h03

9/ AnimateDiff – appends a motion modeling module to a frozen text-to-image model, which is then trained and used to animate existing personalized models to produce diverse and personalized animated images. https://t.co/L4VOSdideP
— DAIR.AI (@dair_ai) 16 juillet 2023

9/ AnimateDiff – appends a motion modeling module to a frozen text-to-image model, which is then trained and used to animate existing personalized models to produce diverse and personalized animated images.

LLMs as General Pattern Machines: In-Context Learning and Robotics

By

–

16 July 2023 16h03

6/ LLMs as General Pattern Machines – shows that without any additional training, LLMs are general sequence modelers, driven by in-context learning; applies 0-shot capabilities to robotics and transfer the pattern among words to actions.

NaViT: Flexible Vision Transformer for Any Resolution

By

–

16 July 2023 16h03

5/ Patch n’ Pack: NaViT – a vision transformer for any aspect ratio and resolution through sequence packing; enables flexible model usage, improved training efficiency, and transfers to tasks involving image and video classification among others.

→ View original post on X — @_akhaliq

Zero-1-to-3: Single Image to 3D Object Generation

By

@_akhaliq

–

16 July 2023 4h30

Zero-1-to-3: Zero-shot One Image to 3D Object

github: https://t.co/2QCZ1apGdv
demo: https://t.co/Z92zQae2Zn

learn to control the camera perspective in large-scale diffusion models, enabling zero-shot novel view synthesis and 3D reconstruction from a single image pic.twitter.com/dJSBD2qXMY
— AK (@_akhaliq) 16 juillet 2023

Zero-1-to-3: Zero-shot One Image to 3D Object github: https://
github.com/cvlab-columbia
/zero123
…
demo: https://
huggingface.co/spaces/cvlab/z
ero123-live
… learn to control the camera perspective in large-scale diffusion models, enabling zero-shot novel view synthesis and 3D reconstruction from a single image