hard rain -SD oil paint remix

Studio Artist processed story guided generative image synthesis animation created using stable diffusion variant of latent diffusion.  From a series of posts with the same 'hard rain' storyline with different styles and animation settings.  Time expanding in Studio Artist in addition to temporal smoothing and various enhancement effects.

All of the different animation runs in this post are using the exact same text prompt, same story line and same style.  None of that changes.  The only thing that changes are other adjustable parameters associated with the animation run.

This first one has no previous frame feedback.  The text embedding is interpolated 4 times between each text prompt keyframe.  Using flow transition in Studio Artist to time expand it out to match the other animation run.

The second animation run below is feeding back the previous frame into the stable diffusion model using the RGB coherence animation option.  I used flow transition option for the 2X time expansion in Studio Artist for this one, i think the wobble is from that.  Fade would probably be better.

The third animation run below is feeding back the previous frame into the stable diffusion model using the LAB coherence animation option.  I used fade transition option for the 2X time expansion in Studio Artist for this one, so you can compare it to the previous animation that used flow and a different colorspace for the coherence.

The 4th animation run below is a different text prompt embedding interpolation option processing every frame.  No color coherence monkey business in this one, no spatial warp of previous frame influence.

In hindsight i should have used the same random seed for all of these runs, but they are different, so it shows you the degree of variance you can get for the same prompt input. As the animation progresses, the characteristics of the coherence option being used (or not used) influences the visual appearance of the resulting recursive output more and more as time goes on.  As i have pointed out before, the way that coherence option works is that it ties your animation to the associated colorspace histogram of your first rendered frame, which is oftentimes not what you would want for the entire animation run.

The # of diffusion cycles per frame associated with the recursive feedback (strength parameter inversely adjusts this) is also probably a little too low in this set of feedback runs.  It seems to be having a hard time keeping up with tracking the story line.

We will continue this 'Hard Rain' excursion in a part 2 post where we dial down the strength parameter, thus increasing the # of diffusion cycles per frame to try and get the recursive runs to follow the storyline better.