T.S. Elliot -SD remix

A friend of mine suggested using a T.S. Elliot poem as the storyline for a generative ai animation.  This series of tests is using the first paragraph of the T.S Elliot poem titled 'The Wasteland".  Each line of that paragraph from the much longer poem is used as a story keyframe, along with an additional fixed style prompt.

Recursive feedback of a warped previous frame above, interpolating the text embeddings below.

Of course when you see things like paint brushes and cans of paint in the generative output, you wonder how much of that as well as things like the tonality, the lighting, the actual subject matter, etc are actually a product of the style text prompting rather than the actual text of the poem.  So the runs below get rid of all of the styling prompts and only use the literal text from the poem.

Recursive feedback of a warped previous frame above, interpolating the text embeddings below.

Getting rid of the style prompting gives the generated output a more drab documentary visual look.

Now i've mentioned before how the recursive feedback animation approaches require the use of a color coherence adjustment that maps every successive frame back to the coloring associated with the first frame, which is far from ideal for many reasons, and you can see it in the recursive feedback animation examples above. 

There is a trick you can use to try and overcome that, which involves adding an extra prompt at frame 0 only that forces a more vibrant coloring for that start frame.  So you can see that below using the same story line (just poem text except for frame 0 only which adds 'vibrant color') as the 2 animations directly above.  This brings more color into the entire animation even though the modified prompt was only used at frame 0.

For the experiment below i totally changed the prompting text for frame 0 only to not include the poem text at all, and instead use a totally different prompt to start off the animation.  The poem text jumps in at frame 1 and then the rest of the keyframes.

For the final experiment below i turn the styling on again with a different styling prompt, which i have to be honest seems to give the resulting animation a more poetic feel than is you just use the poem text.