Page MenuHome

New compositor node: keyer
Closed, ArchivedPublicPATCH

Description

Add a full featured keyer node to the compositor.


Pull the matte:
-With garbage matting
-Permit matte of type Matte=Green-Red or Matte=Gren-max(Red,Blue) (works with blue and red background to)
-Permit to input a preprocessed pallte for matte pulling that won't be used for compositing (permit to clamp value of highlights or blur the UV component of a video, ... outside the node )
-Can use the "screen correction" technique to correct nasty platte (with a drak corner in the backgound, or with tracking markers, ...)
-Scale the matte to BW level
-Blur the edge

Despill:
-Permit to limit by ! channel or average of channels
-Permit to "enhance" de despill
-Permit to apply the spillmap to all channels (unspill method)

Color correction:
-This must be aplly internally to the despilled platte, so I had to include it.
-Primary color correction (Lift, Gamma, Gain)
-Secondary color correction (Hue, Saturation, Value)

Compositing:
-Simple compositing or "processed foreground" compositing. This give softer edges, but for this you have to choose a "solid color" (also used for garbage matting) with all channel slightly supperior to the background color of your platte (this does not afect the garbage matting to work well)
-Edge blending blur (edge detection + refining of the edge matte + blur the edge of the composite)

I tried to keep the interface small and simple but still, the node is pretty big.

I tested it intensively and it really can gives amazing results on almost any screen. Of course blender could do this allready but with about 100 nodes and a lot of redondant/not used input that confused a lot.

I allready wrote some documentaion on the internals and the use of the node, but I have no online place to put it. But basicly it permits all the methods described in the book "Digital composting for film and video" by Steve Wrights.

I hope this will be usefull :)

xavier

Event Timeline

Sorry I forgot to add a file to svn before creating the patch.
So here is the complete one.

xavier

This looks fantastic! Keep up the great work.

juho, assigning to you.

I am reassigning this to Robert Holcomb as I believe he was the one that implemented Blender's current matte solution in the first place. Furthermore my knowledge about matte techniques is pretty much nonexistent.

I saw your documentation at the wiki (http://wiki.blender.org/index.php/User_talk:Xat). As mentioned, the user interface is pretty massive. Furthermore it has some communication issues (I admit that there are these sort of issues in Blender already :) ). A good example of this is the row with Inv buttons and the color pickers above. Matt Ebb can probably lend his insight on this issue.

Perhaps we need to add some way to label and fold parts of node? (I made concept of this earlier already for sockets. This might be something nice to work for 2.50 or for some later version?)

This may sound stupid as I don't know how the node really works but would it be possible to split it in a couple of nodes each representing part of the functionality? This might help to alleviate the size/communication issue as well.

Hello, thanks for the attention.

Yes the user interface was my frist preocupation so I tried to keep it simple, but a keyer have lots of parameters. But note that I took a screenshot with all the option unfolded, normally just the used one are showed.

For the Lift Gamma Gain control (and the related invert button) I stay consistant with the color correction tool of the sequencer (but I can change it).

It is possible to split the node, but the all in one has some advantage:
- You almost never do a greenscreen without despill and color correction and compositing.
- This must be apply the the good plattes in the good order.
- There are intermediate steps that must be insert at the right place for some technique (for the processed foregroung compositing method, the foreground must be processed before the despill oeration)
- If some internal tools does not suit you (can be the color correction) you still can get the intermediate results a do some steps out of the node (and unecessary calculation are ignored)
- The edge detection filter node is buggy and give really too much sharp result to be usable as is for edge blending blur after composite.
- Its quicker

But I stay open, and I agree that a smaller UI would be best.

Here is some things I tought of:
-I could detect the baking channel automaticaly from the dominant channel of the "solid color" used for garbage matting, this will remove the 3 R G B button at the top.
-The 3 R G B option for matte pulling can be reduce to 2 by automaticaly removing the backing channel (this is not consistant with other nodes)
-The edge blending blur "enabling" button can be removed, I can check if I have to do the calculation by checking if the last "size" parameter is supperior to 0, but I found it more "communicative" to hide and show the parameters by enabling/desabling the main button.
-Same for "unspill button".

Has I said, I am open to suit any of your requirements (after all I am just a small hobyist developper and you are the gods that developped my favorite soft for years).

Will add a screenshot of the node with minimal UI to show you how it can look when not fully used.

xavier

That looks cool! thats not a very hard example tough. Have you read this technique? I assume you are using a similar system

http://www.blendedplanet.com/?Freebie_Tutorials:The_Ultimate_Keyer

Thanks,

I put a simple example from a wikipedia picture because the previous one had license issue.

I read the technique you link to and is very interesting. Mine do not work really in the same way.

My node doesn't do any "magic", it just permit to switch easily between the principal basic methods used for creating matte:
Matte=Green - Red
Matte=Green - max(Red, Blue)
...

The link you mention use such a technique and an other one with better edge but not full density in the foreground area to create a hybrid matte. You can still do such a comp with my node and the "soft comp, hard comp" technique (first do a composite with the soft but not full density matte and use the result as background to composite with the hard matte which fit "inside" the soft one ).

Note that it is possible to add different method to the node, but the "soft" one of the link will also need more parameters to be added and maybe more outputs to permit inspection of the intermediate image (sampling values from intermediate image is often the best way to figure out the best value for the parameters).

My node also does the despilling and color correction which are not mentioned in the link.

It also produce much better/quicker edge detection for blurring the resulting composite:
-The filter node produce result 1 pixel shifted to the left so the edge matte does not superpose the edge exactly (patch in the tracker to correct this)
-It also produce a hard edge matte with value up to 4.0 which is difficult/impossible to scale to the desired size with blur and scale operation (need a scale, blur, scale operation). (For a 2 pixel wide soft edge, you need a 2 pixel wide edge matte with soft edge, difficult to obtain from a 4 pixel wide matte with hard edge)

All the magic that can really help to get better matte can be done trough the "preprocessed plate input" but this is really dependant of the technique (film or video) the compression, the lightning of the scene, the cloth of the subject, and much more.

I am no trying to do a "magic" node but to simplify the workflow with a flexible node providing fully predictable results.

xavier

New patch to avoid clamping to 0 or 1. Now it just clamp sub-black value to 0 and only if the "processed foreground" compositing method is used.

Hello,

I reworked the node to split it in smaller ones:
- A Greenscreen node that o the matte pulling operation
- A Edgeblur node that blur an image along the edges of a mask
- Improved the existing Colorspill node

This brings less massive UI and the possibility to use several "greenscreen" nodes and mix the resultings matte.

Note:
Color correction is gone
Lightmap node to come
Need info if the DNA changes for the colorspill node are fine so I can make the do_version

Nice work, i put a windows build with your patch on graphicall.org
http://graphicall.org/builds/builds/showbuild.php?action=show&id=969

:-)

Hi All,

I downloaded the windows build from graphicall.org
Having tried this with the examples from the book cd I
can say that it produces fantastic results without the
need for a mass of noodles etc the results are on a par
with the best and should be incorporated into the std biuld
asap.
However I have tried to do my own build using the latest
release and the edgeblur throws up error saying that custom 3&4 are not part of bnode DNA_node_types for me has only
custom 1&2 there
Keep up the good work I hope this helps.

Mick.P

p.s. A slice graph would be a bonus :)


Xavier you absolutely rock. This will be of huge importance when Ton gets around to the live-action open movie project.

If you are implementing this sort of a useful function, perhaps you would be willing to look at http://users.utu.fi/jhkarh/oflow.html. Blender is desperately in need of an optical flow algorithm for time remapping. That is until libmv gets mature enough to build on top of and build structure from motion for a more acute time remap.

Great work. Is there anything you require testing as I have copious quantities of various greenscreens and bluescreens?

I think now that 2.5 is underway we should incorporate these nodes into blender. I'll take care of the merge later today. I will look at replacing the existing despill node with this new one.

Xavier, this is great work.

Hy Robert,

That is great, last year when I finished this and wanted to do some color manipulation node Ton said that these kind of nodes where too specific and that I should wait for nodes to be plug-gable in C. However Matt recently coded the nodes I was thinking of (Lift Gamma Gain and Hue curve correction) into trunk. So I guess the situation has changed :)

Regarding integration into 2.5, note that was my first peace of code for blender and I never got a review so I have no really idea if this is "good" code.

Regarding the despill node, it is quite easy to integrate with the original one: it is equivalent of using my node with "average limit" and not using "unspill" the "ratio" and "enhance" parameters conversion is:
ratio = 1 - enhance

I also have some free time for the moment, so if you want I can start porting the nodes to 2.5 and you can review. This way I can maybe learn how to become more experienced in blender dev.

Updated link to the initial doc I made:
http://wiki.blender.org/index.php/User:Xat/KeyerNode

Feel free to contact me by pm or IRC.

Thanks

Xavier

Incorporated parts of this patch. Specifically, updated the current despill node with this one, and the channel keyer node with one of the algorithms present in this one.

Robert Holcomb (scourage) changed the task status from Unknown Status to Unknown Status.Mar 15 2010, 4:23 PM