The big improvements over the first generation version of this from last year (
https://
app.eden.art/creations/65b1
5634c7e9dd341fd49258
…) are: 1) More distinct style. I got a bit tired of that AnimateDiff aesthetic on top of every film. The IPAdapter trick helps here. 2) Multi-stage writing pipeline is great
MULTIMODAL AI
-
Improved AI Video Generation with IPAdapter and Multi-Stage Pipeline
By
–
-
AI-Generated Film Production: Storyboard to Music Composition
By
–
Last but not least, the music! A prompt is generated at the storyboard phase, which then goes to Musicgen to generate a backing soundtrack for the entire film.
— Gene Kogan (@genekogan) 8 septembre 2024
Videos, vocals, and music are all composited together, and voila. pic.twitter.com/EbngkyR2BzLast but not least, the music! A prompt is generated at the storyboard phase, which then goes to Musicgen to generate a backing soundtrack for the entire film. Videos, vocals, and music are all composited together, and voila.
-
IPAdapter Image Generation Shows Unique Style Bleed Effects
By
–
Another example, this style image: Generated these frames. There's some subtle content bleed from the IPAdapter, like the random tree branches everywhere, but I like it — feels more unique / idiosyncratic.
-
Converting Screenplays to Storyboards with AI Voice Generation
By
–
Then convert the screenplay into a storyboard. The storyboard is a sequence of images that retell the screenplay, along with character dialogue. All the characters are assigned matching voices from ElevenLabs, using their description as a prompt. Vocals are generated, giving us
-
AI Pipeline Generates Short Films from Text Prompts
By
–
I've been experimentinmg with pipelines for generating short films / trailers from prompts.
— Gene Kogan (@genekogan) 8 septembre 2024
The process goes roughly like this:
1) Synopsis: draft a basic story kernel and visual aesthetic.
2) Storyboard: Expand synopsis through multiple stages (script/cast -> screenplay ->… pic.twitter.com/OAYeGCgSCLI've been experimentinmg with pipelines for generating short films / trailers from prompts. The process goes roughly like this: 1) Synopsis: draft a basic story kernel and visual aesthetic.
2) Storyboard: Expand synopsis through multiple stages (script/cast -> screenplay -> -
Flow speech synthesis tool needs fine-tuning for vowel accuracy
By
–
Flow is what I use now. It's pretty good, but gets my vowel sounds wrong sometimes. I'd love to be able to use a fine tuned model.
-
Interactive AI-generated story features using DALL-E 3
By
–
Stories will appear in a form of an image with a text overlay You will be able to browse through its chapters and it will continue being generated as you swipe! It seems like images are generated with Dalle-3. In general it is an interesting twist assuming you can also
-
GameNGen: The Sora Moment for World Simulation
By
–
The more I read about GameNGen, the more I think it's groundbreaking. I mean @hardmaru @SchmidhuberAI were definitely the OG of this idea, but this is the Sora moment for world simulation. I think of simulation as the terminal goal of the entire field of generative AI. Every
-
Google and Anthropic Launch Grok-2 Cluster and Video Generator
By
–
Big News: Google & Anthropic are making waves. Grok-2, the world’s most powerful training cluster, and a new video generator have just been introduced. And that's just the beginning! #Grok #X #AI #Google #Claude #LLM
