Movie Mosaic Experiment 4

 

As promised, i added a new feature to pre_V6 so that you can work with free running movies by just dropping them in a folder and using them in an image folder brush.  Constructing a sub-nested stack of movies in a single movie brush is pretty tedious, so if you don't need 2D index modulation for some specific purpose this new approach is way easier to work with.

Quick second test with free running movies via image folder brush in a simple time particle paint preset.

the masque of youth 2

 

Movie Mosaic Experiment 3

 

I reworked a few ass-backwards bug issues in the 2D index modulation for movie brushes that apparently fell through the cracks in V5.5 testing.  So this run is using AnimFrame Cycle Fwd for 2D Mod A and RGB Mapping for 2D Mod B.  Which is how you would really want to specify it if you want the back to back movie sections in the single movie brush to RGB map into a specific sub movie for coloring (the B axis modulation) and the frame advance within that specific sub movie (the A axis modulation).

It would be ideal to speed up how this whole things works, and add a few more feature options to make it more flexible, so that will happen today.

Now that things are working properly again with 2D modulation, you can run some wacky experiments like the one shown above, with 2D Mod A =  RGB Mapping, 2D Mod B = Lum, and the Attribute Index remapped using the Luminance option.  The attribute index remapping trashes the sub ordering of the 44 sub movies stacked end to end across the movie brush, since all of the movie frames in the entire movie brush are luminance indexed now.  The B modulation luminance indexes into a specific 100 frame sub segment of the luminance mapped frames, and the A RGB modulation runs through those 100 remapped frame images and picks the best one for the local coloring.

If you swap the A-B modulation indexing for the first photo mosaic setup shown at the top of this post, this is the correct behavior you now get.  We're using A = RGB Mapping and B = AnimFrame Cycle Fwd.  The B indexing is moving through the sub movies one by one, and the A indexing selects the best frame in the sub movie to represent the coloring of a local area.  I just ran a few frames in the movie below to get across what i'm talking about.

Movie Mosaic Experiment 2

 

Normal movie brush running with the animation frame indexing in a paint synthesizer mosaic preset.  The movie brush is used as a source brush to apply solid paint to the canvas as opposed to the previous example where the movie frames were the paint nibs.  

Below is the same movie brush used in a time particle preset.

the masque of youth

 

Movie Mosaic Experiment

 

Experiment using 2D index modulation with an interleaved movie brush setup so that the individual movie frame advance in the photo mosaic.  The old ArtMapper program let you do this kind of thing as one of its more advanced features, and a question came up on the Studio Artist User Forum about emulating it in V5.5.  44 different movies of 100 frames each  back to back in a 4400 frame movie brush.  I'm going to add a new feature to pre_V6 to make this kind of thing trivial to construct and experiment with, since this old 2D index modulation approach is pretty tedious to put together.

underwater ballet 2

 

There are all generated using a static text prompt the CompVis Pytorch code modulating the U-Net latent input in different ways to drive the animation movement.