 keying in the 2nd dimension. A while ago, I came across a post on Eran Leroy’s blog that blew my mind. He demod a 2d keyer using STMaps to generate a point cloud that could be masked with a roto shape to output an alpha. I think this method has potential to get some use in keying a variety of subject matter, so I decided to package up the demo from the blog into a functional keyer. Nuke primarily has three types of keyers, 1-dimensional, 3-dimensional, and algorithmic . 1D keyers create an alpha channel from a single channel (LumaKeyer), algorithmic keyers work by comparing two channels against each other (Keylight), and 3D keyers plot one channel per XYZ axis and define a 3D shape for the keyed areas (Primatte). Nuke pretty much has it all for keying, except for the 2-dimensional keyer. This tool takes keying to a whole new dimension. A quick note: the core of this tool revolves around using STMaps, Ben McEwan has a good write up on how they work if you need a refresher. instructions.
some definitions. Before diving into the guts of the group, it’s important to have a surface level understanding of a few key terms. Color space is a mathematical interpretation used to describe the color of light, displayed as three numerical values. Luminance or value can be described as the “brightness” of light i.e. dark blue vs light blue, we’re talking about the different shades of light in the blue, not the color.  Chrominance refers to how intense the hue and/or saturation of a given color is. Colors with low chrominance are “weak” while colors with high chrominance are vivid. Chrominance is often, but not always, described using two color channels.  STMap is a Nuke node that moves pixels around in an image based on a two-channel UV map. The two channels correspond to the (x, y) “absolute position” of each pixel in the image. The values in the UV map are between 0 and 1 where (0,0) is the bottom left of the image, and (1,1) is the top right. The STMap node moves the pixels from the ‘src’ input based on the values of the two-channels in the ‘stmap’ input. For those that understand written code better than words, here are the workings of the STMap node in a BlinkScript kernel that might help explain what’s happening under the hood of the STMap node a bit more.  separate luma from chroma. The three channels that make up a layer inside Nuke can represent various components of color as well as other types of data. Many familiar color spaces store the amount of red, green, and blue light in their respective channels (linear, rec709, log, etc.). The color spaces most useful for this tool are those that store the luminance information separately from chrominance information. An example of such a color space is one familiar to many artists, HSV, which uses an individual channel for hue, saturation, and value (luma, sometimes called brightness). The default color space used in this tool is the YCbCr color space, where Y denotes the amount of luminance (brightness), and Cb and Cr the blue-difference and red-difference, respectively. Color science is a large box that we don’t need to dig deep into for this tool; the important information to hold on to is that certain color spaces let us separate two-channel chrominance information from single-channel luminance information.  The image above shows the components of the YCbCr color space that work to create the RGB image on the left. The Y channel is solely the brightness of the overall image. The Cb circle shows the amount of blue chrominance in the image. Note the area near the top left contains values closer to white (lots of blue chrominance) while the bottom right displays values closer to black (no blue chrominance). The Cr cirlce shows the amount of red chrominance in the image. Again, note the areas of white vs black. Finally, note how the greens are represented in this color diagram. Green has very little chroma red or blue but lots of luminance. This will help us plot the chroma information and draw an alpha channel later. the basic setup.
generating the point cloud visualizer. We can plot one channel of chrominance on the x-axis of the viewer and the other channel of the chrominance on the y-axis using an inverse STMap BlinkScript kernel that Mads Hagbarth created and Erwan Lery adapted for use in this tool. This is the reverse operation as the STMap kernel posted above.This kernel generates a 2D point cloud that maps the source footage based on the Cb and Cr chroma channels. The x-axis of the point cloud represents the amount of blue-difference (Cb) in the image, the more “blue” a color has, the further on the positive x axis the pixel will be plotted. Similarly, the y-axis represents the amount of red-difference in the image, the more “red” a pixel has, the higher on the positive y axis it will be plotted. This leaves pixels with high values of “green” to remain closer to the origin. This point cloud separates out the screen color of the green (or blue) screen from the rest of the foreground, which makes masking the screen colors with the roto shape much easier.
However, there are still a few quality of life adjustments that can be made to the point cloud. Firstly, I’ve added two different shuffle nodes for swapping the axis of the point cloud. This could be helpful in separating out the chroma values for easier masking. Secondly, there’s an added grade node that’s used to transform the point cloud. The blackpoint and whitepoint can be used to scale the point cloud up and down, as needed. The offset value of the grade will translate the point cloud in the x and y. Tertiarily, I’ve added a simple blur and premult to add more density to the individual point cloud pixels. Finally, my favorite addition is the ability to overlay the point cloud over the source footage. This makes fine tuning the alpha channel a much smoother experience as the user can see the point cloud and the output alpha at the same time. downloads. I hope this tool finds a home in the Nuke artist’s ever expanding repository of keying techniques. This is definitely one of the more unique ways to manipulate pixel information that I’ve come across and it’s been an exciting adventure dissecting the inner workings of STMaps.
Happy comping!
1 Comment
|
