This is the first part of our four-part series on how to freeze a person.
Going from pre-production to keying, 3D camera projection to tracking and to finally comping the parts together, we will guide you through the creation of the following effect:

We could dive right into the process of keying in Nuke (the Foundry), but – wait! Stop it! Let’s take some time to talk about the preliminary stuff. The greenscreen footage is the most crucial part, so good planning is everything.
When shooting for VFX you always need to keep in mind the material is going to be worked on a lot, so it is your job to keep the post-production work as low as possible. In a real world production this equals to saving tons of money.

The first rule is: KNOW YOUR SHOT. Before even touching the camera, you should know what you are going for, what elements the scene will include (both props and CG-elements) and how the camera is going to move. Also you want your shot to look as close to the desired result as possible. Forget » meh, we’ll do that in post. « – if the light is supposed to look red in the final shot, use a red filter on set.

You might have all the ideas yet, but a good storyboard is the best start to go for. Not only does it help you keep track of the work on set, but it is also a way of communicating your ideas to others involved.
We decided to shoot on greenscreen, as we knew we would need our protagonist as a separate element and we wanted to put in our own background.
Using our VFX Vector Package, we’ve created a draft plan of what we wanted our setup to look like:
As you can see we created a dramatic lighting consisting of a reddish keylight, a blue rim light and an additional subtle fill light. We shot our footage with an EOS 5D Mark II because of its full-sized sensor and (thanks to the new firmware) 24fps at a full HD resolution. We thank Boris for his contribution as actor!

Let’s key.
In case you wonder some features might not be applicable in your Nuke version: we are using Nuke 6.
For the sake of briefness, our tutorial will imply a basic understanding of Nuke’s workflow. If you are not all familiar with the software we encourage you to look at the sections Basic Workflow and General UI provided by The Foundry.

Before importing our footage, we used After Effects to convert it to a TIFF sequence, since Nuke works best with image sequences.
Now start by opening a new file and changing the project settings to 24 fps and a resolution of 1920×1080 by hitting on your keyboard.
Hit to import the green-screen footage and connect it with a viewer. Before everything else, we had to degrain it using a DegrainSimple node. Walk through the channels using , and while adjusting the settings of the node until you get out most of the grain.

Having the perfect key with only one keyer is very rarely the case. More likely, you need to extract the hair seperately and mix two to three different keyer together. A garbage mask around the subject to narrow down the area to process, a core matte making sure his body doesn’t get any transparencies or gaps, and the main key itself.
We pulled a simple Primatte Color Key for our core matte, which was then eroded by 11 pixels using an Erode node. Never forget to premultiply the result with a Premultiply node, for we will need the alpha channel to be baked in before proceeding.
To get a clean matte for the hair, we created an IBKGizmo node (hit in your node graph and enter the desired node name to create a new node) and changed its screen type to pick. For the color property use the eyedropper to choose a color in the viewer. Since we just took a sampled area, the color might not be accurate for the whole screen, so opening up the color sliders window helps to do small adjustments. Play with the red and green weight until you get a satisfying result: the hair shouldn’t be too smooth or dilated, but neither should they have a big fringe from the green screen. We added a Grade node and played around with the blacklevels to even improve the matte.
Again using add a Copy and connect the IBKGizmo (or like in our case, the Grade node) to the A input of it. Then connect the original footage to the B input of the Copy. We added a HueCorrect and another Grade and tweaked the settings to get a better result. Premultiply. Your node tree should now look like this:
The key you’ve pulled might not look perfect, as small holes like the following will show up – don’t worry, that’s what we are having our Primatte core-matte for.
As you can already start to see in the alpha channel, our keyers tend to extract colors inside of his outline too – that’s why we need our core matte. Don’t forget to add some Backdrop nodes to name the different sections of your stream. We tend to color everything greenscreen-related in lime.

For our garbage mask we used the Bezier Tool in a RotoPaint node (hit ) and roughly masked the main character out.
If you plug your degrained footage into the bg-input of the RotoPaint you can place your points more easily.

To combine the key we pulled on the hair and the Keylight keyer used for the rest of the body create a Keymix. Use the hair-matte on input A and the Keylight on input B. This will tell Keymix to place A over B.
As all we need from input A is the area around the hair, we went into Imagineer Systems’ Mocha to pin a simple shape to his hair. It is up to you whether you want to let Mocha calculate your hair matte or if you just want to draw it yourself in the same manner we did the garbage matte.
In Mocha, we used the Bezier Tool to draw a rough shape covering all the hair but leaving out his ears and set our Min % Pixels Used value to about 3 before we started the tracking. If you want to learn more about the workflow of Mocha, go ahead and take a look at their tutorial section here.

After Mocha finished tracking, click on the Export Shape Data… button, choose Nuke Roto Paint and then press Copy to Clipboard.
Go back into Nuke and paste the copied roto data. It will insert a new RotoPaint with the tracked shape in it – that is what we hook onto our Keymix, using it as our mask input. The result is a matte using the IBKGizmo to extract the hair and Keylight for the rest of his body.

If you look at the image a little closer, you’ll see that we still have a problem with our tracking markers. Our key won’t catch them, and our garbagemask will only get rid of them as long as they are outside our rough track. Looks like we need to roto them out by hand, so lets start with a new roto-node and use the keymix as background input since this node already has the garbagemask included. Take the pen-tool and start painting our points out. A little trick is to set the Next increment to 4, so you can jump 4 frames, rearrange your roto, and move forward another 4 Frames.

Since our key works as an include-mask and the roto is an exclude-mask we need to invert it to match the keymix output. To do so create an Invert Alpha node and a Merge node to merge our rot with our keymix. Also you can disconnect the background from the roto, since we don’t need nuke to calculate this again. This step should leave you with something like this:

Create a Copy node hitting and copy the merged keys into the alpha channel of our Keylight result. If your result shows weird pixels that’s because we haven’t premultiplied yet. And we keep it that way, as we are not done with it yet. This is what your node graph should look like by now:

Yielding this result:

As his movements slow down due to the freezing process we retimed the original footage using the Time Remap filter in After Effects.

Learn about the creation of our background, camera projection and the importance of correct focal length in our next tutorial – here!

Jake & Dan

...you might also be interested in:

• Freeze Effect Part 4/4:
  The Snow and The Ice
• Freeze Effect Part 3/4:
  Integrating The Elements
• Freeze Effect Part 2/4:
  The Backdrop