public hanging - fear of flying

 

Another generative ai image synthesis animation experiment using the lyrics from a Public Hanging song as the content text prompting (the lyrics key-frame as the animation progresses).  Working with fixed style prompts for all of these (no key-framing).  Song lyrics that are allegories for an emotional journey that work well in a greater musical context seem to be difficult to pull coherent imagery from with this particular generative synthesis approach (which is extremely literal in terms of the imagery it generates).  Maybe we need a new CLIP model that works with multi-modal emotional similarity (existing CLIP is all about multi-modal text-image description).

public hanging - strange disease 2

 

Continuation of a generative ai image synthesis animation experiment created using the song lyrics from the Public Hanging song 'Strange Disease'.  The vagaries of memory are strange, because i remember all the lyrics perfectly, but i may be butchering the official title used on the original cassette release.

public hanging - strange disease

 

Continuation of the generative ai image synthesis animation experiments created using the song lyric text from old Public Hanging songs (a band i was in back in the 80s in Portland Oregon) as content text prompting input for the multi-modal generative image synthesis algorithm.  There is a separate style prompt input associated with the generative algorithm that is messed with in different ways for the different animation runs.

The particular song this generative animation experiment is based on was about living with the existential threat aids back in the 80s.  The song lyrics seem to ring even more true during the covid pandemic era.

public hanging - dangerous toys

 

Continuation of some generative ai image synthesis animation experiments created from the song lyrics of some old Public Hanging songs.  This one is using the lyrics of the 'Dangerous Toys' track on the 'War Zone' cassette release from 1986.

This first run is probably a classic case of the imagery content being taken over by the style prompting.  So i tried to push it (probably too far) in a different direction for the second run below.

Whenever i hear this particular song i always feel like i'm moving through some imaginary evocative landscape, so i tried to push it in that direction.  Probably too grand canyon literal above, so i tried something with more of a murky edge below.

public hanging -peace with honor

 

An old friend of mine that i was in a band with in Portland Oregon in the 80s called Public Hanging (think art gallery but at the same time the name messes with your head when you see it on a poster) recently got in touch with me and i thought it would be fun to try something similar to what i was doing with WMF cd song titles, but to use the actual song lyrics for the text prompting for some generative ai image synthesis animation experiments.  So these are all using the lyrics from the song titled 'peace with honor'.

One thing i find fascinating is that none of the imagery seen in the first example above is actually referenced or mentioned at all in the song lyrics, which you could kind of think of as an allegory of an emotional journey where you realize beliefs you have been fed by society over your lifetime are actually lies.  So no military hardware, no dragons, etc.  I am fascinated by various artistic shading effects that can be created by these generative image synthesis algorithms, so the term 'airbrushed shading' is used in the style section of the text prompting, and i think that is what ends up pushing it into a certain kind of fantasy military video game imagery. 

I got rid of 'airbrushed shading' in the style section of the prompting and tried to push it in a different 'stark' direction.  I was thinking last night i pushed it to far so i tried to dial it back below and ended up drifting back into mercenary robot soldier territory (why)?  I'm liking the stark run more this morning.

I kept trying to pull back air brush style shading effects i was interested in with the 4th animation run below.  It definitely drops back into the fantasy style imagery at some point, although it does capture a few specific lyrics references like the one shown below.

Having the ability to split out how style vs content text prompting actually effects the image synthesis algorithm, using some kind of disambiguation feature for the attention mechanism in the transformer attention part of the latent diffusion synthesis U-Net part of the generative synthesis would be a really useful feature.  The way this particular generative synthesis system works now you get this contamination where what you would really like to effect only the image stylistic character ends up dramatically effecting the content.  So if you specify some characteristic of shading effects you want, it doesn't kick you into a completely different content range, like the current algorithm does.