facial animation in iclone
Mike Sherwood describes how to get the best out of iclone’s powerful new facial animation tools
Improve your work with iclone 7’s new facial animation tools
this tutorial will take you through the many tools and techniques available for facial animation in iclone 7 – including lipsync, keyframing, puppetry and mocap – as well as timeline editing. I'll also describe ways to mix the different techniques in order to improve workflow, as well as how to fix and refine animation at any point in the timeline.
Working purely in iclone 7 with a pre-built character creator model that has been additionally polished in Photoshop and Sculptris, the tutorial will apply to any iclone/cc characters, or any non-standard human characters that have been brought into iclone and correctly set up for facial expressions via 3Dxchange.
When used right out of the box, iclone 7 provides users with a uniquely comprehensive toolset for facial animation that can be used on many levels – from the rapid 'blocking in' of expression keyframes, through to subtle face puppetry and motion capture.
Probably the most important thing to understand is that these techniques can be used separately or together, depending on the results you're aiming for. You can produce decent animation extremely quickly just by using one of the tools, or you can spend time combining different tools and refining the animation, right down to the nervous twitch of a character's lip or eyelid.
For more information about iclone and other Reallusion products, pay a visit to www. reallusion.com.
01 understand the timeline
All facial expression animation is handled on the character's expression timeline. This has four tracks – a clip track at the top, then muscle, eye and head key frame tracks respectively. Select and drag clips or keys to move them, or select and right-click to edit. Check Loop or Speed on the timeline toolbar to loop, speed up or slow down a clip by dragging its right hand boundary.
02 edit expression clips
Adding keys or recording animation to an empty timeline creates a new expression clip. If you right-click on the clip you will have access to all of the usual edit functions, including Break, along with Flatten and Sample clip commands. Flatten embeds all visible keys into the current clip, so you can add more keys cleanly, while Sample generates keys on every frame. Break auto-samples, so for clarity I would recommend flattening immediately before or after breaking.
03 perfect Auto lipsync
Use Create Script on the Modify> animation Panel for auto lipsync methods, which load audio and viseme keys into the viseme track. Keys can be moved, also edited via the right-click menu. Left-click on a key or a track to open the Lips Editor to replace or create a key, and to adjust strength. Select a range on the Lip Options track and right-click for global adjustments.
04 FAST Face key expressions
The Face Key Expression Panel is a library of expressions. Use the top-right drop-down list for different sections. Build facial animations by scrubbing along the timeline and setting keys by double-clicking the desired expression. To establish expressions for naturalistic animation, set an expression key, set again further along the timeline before the next expression.
05 the expressiveness Slider
Use this slider to make expressions more subtle. You can also set expressiveness keys by adjusting the slider between existing keys. This control is on all Face Key Panels and has a global effect, so if you've used expressiveness and want to set keys in the same timeline area without the current value affecting the new keys, flatten the expression clip before adding new keys.
06 refine with Muscle key
It is possible to build up complete facial expressions and animations using the Muscle Panel alone, but it's especially powerful when it comes to adjusting and refining existing expressions, as well as between keys. Single or multiple parts of the face, as well as head, eye and jaw rotation, can all be selected and manipulated simply by left-clicking, holding and then dragging the mouse in different directions from the black background area of the Feature Selection Panel.
07 Default key AND zero keys
The Muscle Panel Default Key button sets muscle, eye and head keys on the current frame to the default clip state. This will be neutral if the clip has no flattened or recorded animation. You can also set keys in the muscle, eye and head tracks simply by double-clicking in empty frames – use these to establish the current timeline state, eg before/after applying expression or rotation keys.
08 Access All AREAS with Modify
The Modify Panel sliders give you access to all underlying morphs, in addition to jaw rotations used by the Expression and Muscle Panels. You can see the sliders in operation each time you play or scrub through a timeline with keys in place. Use the sliders to achieve precision fine-tuning of the existing keys, as well as between keys. This process is extremely useful for error correction. Remember that the Modify sliders also accept negative numeric values if needed.
09 Face key workflow
You could build up an animation solely using Modify sliders, but this would take ages! It’s far quicker and easier to start with blocking in expressions, then refine using Muscle keys and finally Modify sliders. If you find that the clip is becoming cluttered with keys, or expressiveness values are limiting what you want to do, simply flatten the clip and continue adding and editing keys as before.
10 Face puppet – Mocap your Mouse
Face Puppet enables you to drive facial animation using your mouse: its job is to translate the motion of the mouse into full or part-face expressions, or any combination of these, depending on what options you have selected. It’s extremely important to remember to preview before recording, so that you get a feel not only of how far you need to go, but also in which direction, so you can ensure you get the animation you want. Use the Strength slider to increase or decrease the effect.
11 Face puppet recording
Press enter to record at half speed, but combine this with By Frame (click the Realtime button to toggle) on the play bar to go slower still for even more control. You can blend animation by recording over an existing clip with ‘Blend data on next recording’ checked – with this unchecked, it will overwrite the previous clip. You can blend or overwrite full or part face expression recordings.
12 FAST Full Face profiles
You can think of Full Face Profiles as the Face Puppet equivalent of Expression Keys, only dynamic. These profiles make it possible for you to create animations very quickly by simply moving the face into and out of the expressions you want at appropriate points on the timeline. There are many options available to you, from character-specific profiles to generic, universal profiles for emotions, with both lips closed (ideal for use during lipsync) and lips open options.
13 head Motion puppetry
Character profiles have head motion active by default – toggle head turns and tilts on and off using the buttons in the top left of the Solo Feature Selection window. Use associated head motion for quick animation, eg previz, but keep it off for precise face puppetry and add head motion in a separate pass either via Face Puppet, Face Key and/or using Edit Motion Layer.
14 the eye have it
iclone’s auto-blink function is ideal for previz but for control, turn this off on the character’s main Modify Panel and create blinks yourself. Subtle tweaks of the eyelids, when associated with focused eye motion, can have dramatic effects even without using additional parts of the face. For eye motion without mocap, use Face Puppet or set a key to indicate a particular focal point, then set this again a little further along the timeline for establishment before changing eye rotation.
15 refine with Solo Feature
Use the Eraser to deselect the current profile, select a Solo Feature and you’ll see thumbnail options below the selection head. These are profiles for different expression variations for the selected part. Use separately to drive an individual feature to refine an animation, or combine by selecting multiple features and profiles. Mix them with Full Face Profiles if you wish.
16 Face puppet workflow
I generally think of Full Face Profiles as a broad brush, while Solo Features are for more delicate work, but there are no hard and fast rules. While you can block in whole sections of Full Face Profiles before refining with Solo Features, you can also work much more subtly from the start, for example, by doing a pass of eyebrow motion, then mouth, then eyes, etc.
17 work with puppet clips
Recording a pass of Face Puppet generates a clip on the expression track in the same way expression keys create a clip. All clips can be moved, edited, looped or sped up/slowed down. When blending (Blend checkbox selected) Face Puppet passes, start at any point, and the new clip which is created will mix seamlessly with any existing clip already present in the same timeline area.
18 Facial Motion capture
The new iclone 7 Facial Mocap plug-in opens the door to true facial motion capture from established third-party developers. Starting with Faceware camera-based technology, facial mocap gives instant results based on tracking the user’s own facial expressions, making facial animation easier and more straightforward than ever. Mocap recording creates a clip on the expression track as usual, and can be blended with existing keys and/or puppet clips.
19 Set up AND calibrate
Even lighting and good camera setup are important for mocap – the user’s head needs to be framed centrally and straight on as if taking a passport photo. Calibration matches your neutral expression with that of your character. You can use calibration to tweak animation results, eg make your eyebrows higher during calibration to make low brows character animation more effective, make your mouth smaller to increase wide mouth effects, etc.
20 Masking AND Settings
You can mask off parts using the Mask Select head for multi-pass animation: that way you can record just the eyes, mouth or head, or whatever combination you like, and use the Record Blending checkbox to choose whether to blend or overwrite the previous pass. Use Strength sliders to exaggerate or reduce the relative degrees of animation, as well as Smooth to reduce noise/jitters. For lipsync, you can record audio simultaneously by using the Audio Recording checkbox.
21 combine the tools
You can mix iclone’s facial animation approaches in any way, and in any order, but it makes sense to consider best workflow, especially lipsync. Adding lipsync later to an animation that already includes the lips or jaw opening can be tough to resolve. So if the animation includes speech, lipsync first, either by auto lipsync or during mocap, before moving on to use other tools.
22 example workflows
Again, there are no rules, but possible workflows could be: A) Auto lipsync (including manual viseme refinement) > face key expressions and/or face puppet full face blocking > face puppet solo feature and/or face key muscle/modify refinement.
B) Facial mocap > if needed, blend in lip options filtered auto lipsync to audio recorded during mocap (particularly useful for adding tongue motion) > face puppet solo feature and/or face key muscle/modify refinement.
23 polish AND Fix
Once your animation is blocked-in, it may be time to stop if it's for previz. But if you're producing animation for production, it's time to start polishing. This is all about iteration – repeatedly playing and scrubbing through the timeline to check for areas which need improvement. Use Face Key Muscle and Modify for fine tuning, combined with timeline editing, for final animation fixing.
24 clip problem Solving
To reset/zero animation for all or part of a clip, use Face Puppet with Blend data… unchecked and record without mouse movement to zero individual features, eyes, head or all animation. Mocap with masking/zero strength recording and no blending does the same. To clear issues without reset, break and delete the problem clip area and either rework it, or simply use clip transitions to blend between the separated clips.
25 Model For Animation
A character's face may look great when static, but could appear blocky with vertices bunching during animation, particularly at the mouth, eyelids and brows. This is due to modelling: for best animation results try to keep the facial mesh balanced, maintaining good vertex positions that relate well to the default morphing mesh for animation.
26 less is More
Finally, with so many tools and possibilities available in iclone 7, it's all-too easy to overdo animation. So always try to keep your focus on what the character is communicating, and what's needed to clearly get that message across. Good animation can speak volumes with just a simple raising or lowering of an eyebrow, and over-the-top expressions are generally reserved for comedy and extreme situations. Please do bear these points in mind when creating animations. •