underwater ballet

 

All of these are using the quantized keras stable diffusion code on a mac.  For the second run below i ran 2 different experiments, the first always working off of the first cycle into the U-Net latent for the subsequent frames, the second feeds back recursively into the U-Net latent.  If you squint and look at the frames in the first animation you can see the spatial position modulation and associated spatial anchoring derived off of that first frame.  The second one is always self-evolving through the different feedback cycles.

a dream of a better tomorrow -2

 

Two generative image resynthesis sets (above and below) created from a single image of a building in Dubai. I used Studio Artist to create transition animations from the set of 22 images for each set.  Those animations were used in the first 'a dream for a better tomorrow' post to drive the U-Net for the stable diffusion generative image synthesis animations there.

Third generative image resynthesis set below made from same original Dubai image with divergence pushed out all the way.  Below is the transition animation created in Studio Artist that was used to drive the U-Net latent input for the second stable diffusion animation shown below it at the bottom.

Second stable diffusion output mentioned before is above.  Another recursive feedback stable diffusion run below for comparison purposes.

parkour

 

Motion and Form

 

resynthesis variations 2

 

a dream of a better tomorrow

 

Above is a straight recursive feedback of the previous generative output into the U-Net latent to build the animation.  This is using the Pytorch CompVis model on Colab using the exact same static text prompting as the previous 'grin' post, which maybe gives you some more insight when i keep wondering why that other implementation has such different visual properties for it's generative output.

I tried a different approach below where i used the new generalize latent diffusion framework in a 2 step process to build a generative animation that i then use to drive the U-Net latent input below with a static random seed to build the below animations.  So it's one approach to use the generalized architecture resynthesis approach to build somewhat coherent animations (after my 'woman dancing part 3' post yesterday wondering if that was worthwhile pursuing).  I'll dive into the details about how i did that in a part 2 post.

grin

 

Two different approaches to use a single EBM (energy based model) paint action step to build a painted transformation sequence off of a series of recursive generative image synthesis images derived off of a static text prompt.  This is based on the Keras stable diffusion model running on a mac, and the generative behavior really seems different than the CompVis Pytorch models i run on Colab, still trying to figure that out (quantization effects?).