Reconceptualizing Video Processing as Painting over Time

 

Paint animation test in Studio Artist V5.5.4 using another example of the world's simplest paint action sequence. 

Two steps. A dispersive action step based on the SmartBlu IpOp. And a slop down paint action step that is the factory Init Paint Synthesizer QuickEdit painting a single paint nib that is size modulated based on Flow magnitude. 

Many people make the mistake of piling on a ton of stuff into the paint action sequence. Because they are totally missing the point that the paint animation effect occurs over time.  Occurs over multiple frames. And by working with recursive processing, you can setup a very simple paint strategy that plays out over multiple frames (creating behavior over time that can be visually rich and complex). 

And suddenly the world 'most boring' paint preset (as our factory preset designer likes to refer to the Init QuickEdit) suddenly becomes quite interesting.

Visualizing Flow in the Paint Synthesizer

 

Putting together some new presets to aid in visualizing Flow modulation in Studio Artist V5.5.  They are also very useful for tweaking some of the finer points of the implementation details.

So this is a pretty simple PASeq that uses an Outline Vectorizer action step followed by 2 paint action steps. the arrows are MSG Live Brush effects. I'm running injection modulation to modulate the angle and size of the MSG Live Brush.  I'm using a Tile Spacing Grid Scan approach in the paint synthesizer for the arrow positioning.

Variation on Motion Experiment

 

Different take on what i was trying in that last post.  Used soft threshold to process the flow magnitude (Threshold IpOP that routes selection to selection).  Forgot to turn on a fixed seed for the Color Simplify IpOp that is motion matted in before the Flow modulated smear paint (would improve the regonization coloring dropout). 

And we need to add smart clip modulation to Color Simplify, turn on raster EFX like we did in some of the other IpOp effects, and have a no-background option so the old canvas shows though if there is not complete coverage with an EFX.

Motion Experiment

 

Paint animation experiment using lack of motion to inhibit painting.  The jerky stuff at the end is associated with the hand held camera being moved a little bit.  It would be interesting to add a motion compensation component to get rid of that kind of stuff.

Wacky Warp Flow Test

 

Trying out some Flow Warp ideas using the paint synthesizer as the warp engine. I reversed the direction on the flow for this one.

Q.E.D.

 

World's simplest 2 step paint action sequence. Both of the paint action steps use the Init QuickEdit with very minor additional adjustments. I turned on facial feature detection as an inhibitor option in Path Start control panel for the first paint step. I used Flow Angle for Path Angle in Path Angle control panel and Flow Magnitude for Blend in Paint Fill Apply control panel for the second paint step. I used the new injection modulation to do that routing for the second paint step.

Optical Flow Injection Modulation

 

Inaugural run paint animation test of the new injection modulation features in the Studio Artist V5.5 paint synthesizer.  Injection modulation is like bus routing in a DAW. In this particular example, i'm routing the optical flow angle to the direction of the paint, and the optical flow magnitude to the path length of the paint. World's simplest paint preset, use the Init QuickEdit and then the 2 manual edits i just said, and i switched it to straight lines rather than curved. And i'm clipping any paint strokes less than 20 pixels long, so they don't get painted.

I slopped down the world's simplest Vectorizer outline technique black overlay for the edging effect.  Then let it run.  You could of course finesse the presets to make this whole thing much more artistic. 

Which i took a quick stab at below, adding a 10%Mix fade to white and a SmartBlur action step to introduce some dissipation into the recursive processing over time. You put those at the beginning, before the painting. So they process the previous output canvas that is never erased.

Fine Tuning the Optical Flow

 I've been working on fine-tuning the new optical flow features in Studio Artist V5.5. I wanted a nice way to visualize what was going on. So i came up with some paint presets that help out.

Weird Programming Bug Fixed

 

This is more like it. The 'fixed' version of the previous programming bug post. And i even learned something useful in the process.

We're going to make this better, but you can see where it's headed now.

I think it's also worth comparing this new approach to an automatic approach using the old bezier morph context. You could make this much better if you hand draw the bezier paths to define the morph. But if you do it all automatically you end up with a situation like this below.

If i had run the menu commands to normalize the paths i could have gotten it somewhat better, but still not like the optical flow approach.

Dense Optical Flow Modulating the Displacement IpOp Effect

 

Now that i fixed the Dense Optical Flow in Studio Artist V5.5, it's actually useful for something more than giggles. The fix is in Studio Artist V5.5.3.  Quick animation test here.  You will be seeing this as a modulator basically everywhere as time moves on.

Magical Horn Pivot Optical Flow

 

Magical Horn Pivot Optical Flow. Yes you saw it here first.  The mysterious shotgun marriage of Dense Optical Flow and Horn Optical flow, working its magic.  Yes it is mysterious.