Monday, September 12, 2022

trip to vegas

Story guided generative ai image synthesis animation experiment created using latent diffusion and then post processed in Studio Artist.  

The first render below shows off how the rendered output (made with recursive feedback of the previous modified frame into the U-Net latent vector) can get 'stuck' on certain spatial content features that prevent it from tracking the story line.  Like getting stuck on the airplanes outside the terminal for too long.  Or during the taxi ride past the various casinos where it gets stuck looking at a big chunk of blue sky.  The positive side of getting 'stuck' is the whole ending sequence, which free association drifts the story line script in a cool and unexpected way.

The second render below was generated with the exact same story line and style prompts using interpolation of the text prompt latent vectors (no recursive feedback).  The key frames accurately track the story line because of this.  The flip side of this approach is that the level of content diversity is usually more limited than what you get when you feedback a modified previous frame into the U-Net latent vector input.



No comments: