Mauna Loa - day 3

 

Active flow is 2-3 days from hitting the Saddle road based on the current rate of flow.

Bit Fishing

 

public hanging - fear of flying

 

Another generative ai image synthesis animation experiment using the lyrics from a Public Hanging song as the content text prompting (the lyrics key-frame as the animation progresses).  Working with fixed style prompts for all of these (no key-framing).  Song lyrics that are allegories for an emotional journey that work well in a greater musical context seem to be difficult to pull coherent imagery from with this particular generative synthesis approach (which is extremely literal in terms of the imagery it generates).  Maybe we need a new CLIP model that works with multi-modal emotional similarity (existing CLIP is all about multi-modal text-image description).

More GS Controllability 2

 

Reworking some of the gallery show vectorizer preference options to provide more controllability (or variability if you want that instead) when automatically creating generative paint presets that use vectorizer embedding.

More GS Controllability

Working on improvements to Path menu and associated paint synthesizer embedding bezier path generation derived off of the Shape technique in the Vectorizer (it always defaulted to Image Shape in V5.5).

Gallery Show vectorizer meta edit preferences now apply anywhere the vectorizer is used in gallery show, not just to the main technique.  So they now flow through to any vectorizer path generation options for GS paint preferences.  This gives you a lot more controllability over generative paint preset mutation than before.

Mauna Loa

 

Night view of the latest Mauna Loa eruption on the Big Island of Hawaii. Lava currently headed into the north rift flow area.  The southern rift is the one people really get freaked out about (because the steepness of the descent means it can travel very quickly towards the ocean if it heads that way).

public hanging - strange disease 2

 

Continuation of a generative ai image synthesis animation experiment created using the song lyrics from the Public Hanging song 'Strange Disease'.  The vagaries of memory are strange, because i remember all the lyrics perfectly, but i may be butchering the official title used on the original cassette release.

public hanging - strange disease

 

Continuation of the generative ai image synthesis animation experiments created using the song lyric text from old Public Hanging songs (a band i was in back in the 80s in Portland Oregon) as content text prompting input for the multi-modal generative image synthesis algorithm.  There is a separate style prompt input associated with the generative algorithm that is messed with in different ways for the different animation runs.

The particular song this generative animation experiment is based on was about living with the existential threat aids back in the 80s.  The song lyrics seem to ring even more true during the covid pandemic era.

enigmatic landscape

 

Generative ai image synthesis recursive feedback sequence below made into an animation with Studio Artist processing then used as input to gallery show processing in Studio Artist for some representative samples shown above.

Gas Lighting 2

 

public hanging - dangerous toys

 

Continuation of some generative ai image synthesis animation experiments created from the song lyrics of some old Public Hanging songs.  This one is using the lyrics of the 'Dangerous Toys' track on the 'War Zone' cassette release from 1986.

This first run is probably a classic case of the imagery content being taken over by the style prompting.  So i tried to push it (probably too far) in a different direction for the second run below.

Whenever i hear this particular song i always feel like i'm moving through some imaginary evocative landscape, so i tried to push it in that direction.  Probably too grand canyon literal above, so i tried something with more of a murky edge below.

GasLighting

 

Grabbed these during some automated testing using gallery show to stress test paint action sequences to hunt down any potential issues associated with some new features.  Source images were all generated using one-shot image resynthesis.

public hanging -peace with honor

 

An old friend of mine that i was in a band with in Portland Oregon in the 80s called Public Hanging (think art gallery but at the same time the name messes with your head when you see it on a poster) recently got in touch with me and i thought it would be fun to try something similar to what i was doing with WMF cd song titles, but to use the actual song lyrics for the text prompting for some generative ai image synthesis animation experiments.  So these are all using the lyrics from the song titled 'peace with honor'.

One thing i find fascinating is that none of the imagery seen in the first example above is actually referenced or mentioned at all in the song lyrics, which you could kind of think of as an allegory of an emotional journey where you realize beliefs you have been fed by society over your lifetime are actually lies.  So no military hardware, no dragons, etc.  I am fascinated by various artistic shading effects that can be created by these generative image synthesis algorithms, so the term 'airbrushed shading' is used in the style section of the text prompting, and i think that is what ends up pushing it into a certain kind of fantasy military video game imagery. 

I got rid of 'airbrushed shading' in the style section of the prompting and tried to push it in a different 'stark' direction.  I was thinking last night i pushed it to far so i tried to dial it back below and ended up drifting back into mercenary robot soldier territory (why)?  I'm liking the stark run more this morning.

I kept trying to pull back air brush style shading effects i was interested in with the 4th animation run below.  It definitely drops back into the fantasy style imagery at some point, although it does capture a few specific lyrics references like the one shown below.

Having the ability to split out how style vs content text prompting actually effects the image synthesis algorithm, using some kind of disambiguation feature for the attention mechanism in the transformer attention part of the latent diffusion synthesis U-Net part of the generative synthesis would be a really useful feature.  The way this particular generative synthesis system works now you get this contamination where what you would really like to effect only the image stylistic character ends up dramatically effecting the content.  So if you specify some characteristic of shading effects you want, it doesn't kick you into a completely different content range, like the current algorithm does.

Movie Mosaic Experiment 4

 

As promised, i added a new feature to pre_V6 so that you can work with free running movies by just dropping them in a folder and using them in an image folder brush.  Constructing a sub-nested stack of movies in a single movie brush is pretty tedious, so if you don't need 2D index modulation for some specific purpose this new approach is way easier to work with.

Quick second test with free running movies via image folder brush in a simple time particle paint preset.

the masque of youth 2

 

Movie Mosaic Experiment 3

 

I reworked a few ass-backwards bug issues in the 2D index modulation for movie brushes that apparently fell through the cracks in V5.5 testing.  So this run is using AnimFrame Cycle Fwd for 2D Mod A and RGB Mapping for 2D Mod B.  Which is how you would really want to specify it if you want the back to back movie sections in the single movie brush to RGB map into a specific sub movie for coloring (the B axis modulation) and the frame advance within that specific sub movie (the A axis modulation).

It would be ideal to speed up how this whole things works, and add a few more feature options to make it more flexible, so that will happen today.

Now that things are working properly again with 2D modulation, you can run some wacky experiments like the one shown above, with 2D Mod A =  RGB Mapping, 2D Mod B = Lum, and the Attribute Index remapped using the Luminance option.  The attribute index remapping trashes the sub ordering of the 44 sub movies stacked end to end across the movie brush, since all of the movie frames in the entire movie brush are luminance indexed now.  The B modulation luminance indexes into a specific 100 frame sub segment of the luminance mapped frames, and the A RGB modulation runs through those 100 remapped frame images and picks the best one for the local coloring.

If you swap the A-B modulation indexing for the first photo mosaic setup shown at the top of this post, this is the correct behavior you now get.  We're using A = RGB Mapping and B = AnimFrame Cycle Fwd.  The B indexing is moving through the sub movies one by one, and the A indexing selects the best frame in the sub movie to represent the coloring of a local area.  I just ran a few frames in the movie below to get across what i'm talking about.

Movie Mosaic Experiment 2

 

Normal movie brush running with the animation frame indexing in a paint synthesizer mosaic preset.  The movie brush is used as a source brush to apply solid paint to the canvas as opposed to the previous example where the movie frames were the paint nibs.  

Below is the same movie brush used in a time particle preset.

the masque of youth

 

Movie Mosaic Experiment

 

Experiment using 2D index modulation with an interleaved movie brush setup so that the individual movie frame advance in the photo mosaic.  The old ArtMapper program let you do this kind of thing as one of its more advanced features, and a question came up on the Studio Artist User Forum about emulating it in V5.5.  44 different movies of 100 frames each  back to back in a 4400 frame movie brush.  I'm going to add a new feature to pre_V6 to make this kind of thing trivial to construct and experiment with, since this old 2D index modulation approach is pretty tedious to put together.

underwater ballet 2

 

There are all generated using a static text prompt the CompVis Pytorch code modulating the U-Net latent input in different ways to drive the animation movement.