I haven’t found an SDXL ControlNet for segmentation. Do you know a good one?
@fofrai
-
i2vgen-xl Video Generation Model Background Consistency Issues
By
–
i2vgen-xl definitely has some issues around maintaining consistent backgrounds – I've seen this darkening of bright areas a few times.
— fofr (@fofrAI) 5 janvier 2024
(Love that movement in the nearby grass though) pic.twitter.com/pWBdwtkt9Vi2vgen-xl definitely has some issues around maintaining consistent backgrounds – I've seen this darkening of bright areas a few times. (Love that movement in the nearby grass though)
-
1.45GB Tile Model Enhances Stable Diffusion Image Generation
By
–
That is essentially what it does, but it also uses a 1.45gb tile model to help it do that. https://
huggingface.co/lllyasviel/con
trol_v11f1e_sd15_tile
… -
ControlNet Tile Only Available for SD1.5 Currently
By
–
I think Controlnet tile is only available for SD1.5, no one has trained an SDXL version yet.
-
RealisticVision XL V3 Multi-ControlNet LoRA Model Released
By
–
I've added a RealisticVision XL V3 model to Replicate with multi-controlnet and lora support: https://
replicate.com/fofr/realvisxl
-v3-multi-controlnet-lora
… Outputs also work great when upscaled with something like Magnific. -
i2vgen-xl video generation model speeds up significantly
By
–
i2vgen-xl is now running faster – @chenxi_jw has done some great work to speed it up. Steps are now also exposed.
— fofr (@fofrAI) 5 janvier 2024
16 frame gen:
50 steps down from ~180s to ~115s
25 steps at ~60s
More steps makes more consistency.https://t.co/CH1PsbwTyK https://t.co/Dgl6FfdYmU pic.twitter.com/ZgAP1Yhj10i2vgen-xl is now running faster – @chenxi_jw has done some great work to speed it up. Steps are now also exposed. 16 frame gen:
50 steps down from ~180s to ~115s
25 steps at ~60s More steps makes more consistency. https://
replicate.com/cjwbw/i2vgen-xl -
Model Optimization: Fixed 8fps Frame Rate Control Strategy
By
–
Aye, there’s no fps control at the moment, it’s fixed to 8fps, but you can choose how many frames to generate. I think there’s also lots of room for optimisation of this model to get it faster.
-
Single Generation Produces 80-Frame 1280x Video with Remarkable Consistency
By
–
Dang!
— fofr (@fofrAI) 4 janvier 2024
80 frames
a single generation
10 second video at 1280x
Remarkable consistency here over the time frame (10 minutes to generate though) https://t.co/Dgl6FfdYmU pic.twitter.com/NDm8rqeWeeDang! 80 frames
a single generation
10 second video at 1280x Remarkable consistency here over the time frame (10 minutes to generate though) -
i2vgen Model Experiment: Character Movement Consistency Across Prompts
By
–
An experiment with i2vgen model – I keep the seed and params the same but vary prompt by tweaking action (standing, dancing, crying, etc)
— fofr (@fofrAI) 4 janvier 2024
Some pretty solid character movement, consistency is pretty good, only the occasional scary distortion.
12 variations. https://t.co/Dgl6FfdYmU pic.twitter.com/rqTeLC8jt9An experiment with i2vgen model – I keep the seed and params the same but vary prompt by tweaking action (standing, dancing, crying, etc) Some pretty solid character movement, consistency is pretty good, only the occasional scary distortion. 12 variations.
-
i2vgen-xl Image-to-Video Model Now Available on Replicate
By
–
Image to video model "i2vgen-xl" is now on Replicate – make 16:9 videos at 1280×704 resolution:https://t.co/CH1PsbwlJc
— fofr (@fofrAI) 4 janvier 2024
Example (which I also interpolated):https://t.co/2Rm3lZxaeI
It also seems to work well up to 48 frames. pic.twitter.com/qbowIuCQG2Image to video model "i2vgen-xl" is now on Replicate – make 16:9 videos at 1280×704 resolution: https://
replicate.com/cjwbw/i2vgen-xl Example (which I also interpolated): https://
replicate.com/p/lvd42ldbrjr6
mkfje2wawohaja
… It also seems to work well up to 48 frames.