|
|
XTRA! |
 |
Re-Writing the Rules | ||
| A New Way to Face Facial Animation |
|
BY TIMOTHY ALBEE The phrase, nothing is perfect, seems to go double for 3D applications. Yet, we all slog through dealing with programmatic quirks from the minor inconvenience to the supremely frustrating. We're artists after all, not programmers, and we're better off not moaning about things we can't change, and just do our jobs, right? Wrong! |
|
Kobayashi Maru In Star Trek II, Captain Kirk described his time as a cadet when faced with a winless simulation, the Kobayashi Maru, his solution was to re-write the rules of the program so there was a successful solution. We can do the exact same thing when faced with our own winless situation of having to muddle through our jobs with software that often falls far short of what we wished existed!
Timothy Albee's Facial Animation (FA) is just such a tool. Created by Mac Reiter, based upon specifics I'd always wanted to see in a tool for Facial Animation and Dialogue Scrubbing, pushed far beyond those initial ideas by a strong working partnership, FA has already received incredible praise from other artists within the Feature Animation Industry! The end result is a tool that makes this one aspect of animation easier and more enjoyable for everyone!. Kaze, Ghost Warrior Many of you are already aware of the proof-of-concept short-film, Kaze, Ghost Warrior (KGW). Created by one person, on two consumer-grade PC computers, in six months of production, I wanted to show that when the best software is used in an intelligently streamlined production pipeline, the impossible becomes not only possible, but also probable! Among the challenges facing me was my desire to have the facial animation of the characters be Disney Feature quality, both in dialogue synchronization and in emotional performance. On Walt Disney's Dinosaur, an animator was given, on average, four days to produce seven seconds of dialogue animation. I'm pretty fast as an animator, and can do seven seconds of that Disney quality dialogue animation in about two days. Even with the dialogue-sparse script of KGW, there was no way I could kick-out that kind of footage in the time required. I had to find another solution. Remember Your Wishes... I've been an animator for a long time, having started my studio career as a Traditional Tnimator, drawing on paper and working from Exposure Sheets, (X-Sheets). I remember those first, frustrating days learning 3D animation, wondering to myself why we were being asked to do things in what seemed to be the most inelegant fashion imaginable. Rather than suck-up and drive-on, I started jotting down lists of wishes I had for how 3D animation tasks could be made easier, simpler, more elegant. Programs come and programs go. And while some tools had certain elements of what I'd wished for, even the best tools fell short in many areas. As a former Traditional Animator, I'd gotten used to the X-Sheet way of understanding a scene's timing—after all, X-Sheets have been used in the Animation Industry for over 100 years. Curve Editors are handy, but they are a relatively new invention, and more-or-less completely ignore the long-standing conventions held by the X-Sheet. And, as a former avid game-player (and having seen many of the avant-garde demos that pushed the limits of 3D programming), I knew that computer hardware was capable of far more than what most 3D applications delivered. Though this is no real fault of the 3D applications themselves; after all, most 3D applications are calculating many more variables at any given moment than a simple video-game. Speed, functionality, foresight, interface design, minimizing mouse-mileage, not limiting an artist's creativity by limiting a user's options... The list goes on and on. After all, most programs aren't designed by the people who will eventually be using them in a production environment. The Partnership I was first put in touch with Mac Reiter in December of 2003. Mac's GL programming examples were most impressive, and after expressing an initial interest in working together, I put together a mock-up interface of what I wanted, using Flash. As a struggling, independent filmmaker, there was no way I could afford to pay Mac up front for what I felt his skills and services were worth. However, with the way in which this tool could far surpass all current ways of doing facial animation, Mac could easily recoup his expenses in the sale of the finished product. It was a win-win-win situation. I finally would be able to have the facial animation tool that I'd wished for, for so long (and be able to test and use it on the production of Kaze, Ghost Warrior). Mac would have his expenditures covered through the sale of the program when it was released to the public. And the animating public would have access to a tool that would turn the old concept of facial animation on its ear. In Production Production on Kaze, Ghost Warrior and development on TA Facial Animation went hand in hand. Each new scene brought a torrent of suggestions to Mac's in-box. Actually using a tool in production brings to light things that can never be thought of outside a full production environment. Mac's skills as a programmer and his understanding of what goes into feature-quality facial animation really began to show when I'd submit a request, only to find out that he'd already implemented that function in one or two previous builds. And, more often than not, submitting a request for a feature or change (after finding out that something didn't quite work as well in real life as I thought it was going to) would find a new build with those requests fully implemented the following morning. At the beginning of production on KGW, the facial animation tool was letting me do seven seconds worth of dialogue in abut twenty minutes. At the end of production, the tool was letting me do seven seconds of dialogue, often in under six minutes. (The entire final scene of KGW, with two speaking characters, took only one, eight-hour day to animate all the facial animation for both characters.) Facial Animation Animation is the art of revision. The more quickly and easily an animator can revise his work, the better that work will be. Implementing the tools I'd come to depend upon as a Traditional Animator, as well as the conventions I'd come to know as a 3D Animator and putting into practice things I've always thought could/should have long existed, Facial Animation facilitates the quickest creation, evaluation and revision of feature-quality facial animation I've ever known.
Mr. Cool (shown above) comes with Facial Animation, ready to go. You can pull his points to create other human characters' faces (as described in the LightWave 3D Character Animation and Essential LightWave 3D series), retaining all Mr. Cool's Endomorphs in your new character! (And no, that's not a typo—we are getting 50 fps update/feedback on Mr. Cool's head in a Real-Time Sub-D level equal to LightWave's Level 4 SubPatch on a dual 2-GHz Intel running an NVIDIA 4200Ti graphics card.) Better-Than-Real-Time Updates One of the key things I needed to see within FA was to have at least real-time feedback of my manipulations and instant playback of the animation upon request.
Facial Animation gives me an average of 110 frames-per-second feedback on the film-resolution Kaze head at a Subdivision level equal to LightWave's Level 2 SubPatch. True, Real-Time Interaction When I'm evaluating a scene, I want to be able to be able to tweak it without breaking my concentration or chain of thought. FA lets you make most modifications to your Scene, while the animation is playing! (FA even lets you swap heads with similar morph target names without breaking stride!) Undo You do your best work when you aren't afraid of making mistakes. Facial Animation gives you multiple levels of Undo so you can explore your creative ideas with confidence! Real-Time Sub-D
Most character modeling today is done using Subdivision Surfacing (Subpatch Surfaces). Facial Animation does real-time Subdivision equivalent to LightWave's Level 6 Sub-Patch. Symmetrical Splits Most modeling packages now let you work Symmetrically on your model, allowing you to have modifications done to one side, instantly applied to the other. This is an extreme timesaver for creating Morph Targets, but most programs still require you to Split these Morphs yourself into their Left and Right components if you want to work with the asymmetry that is a hallmark of feature-quality animation. FA searches through your Morph Names for indications that a Morph may be symmetrical, splitting them for you along a (user-defined) centerline. These Splits are transparent to the animator, only coming into play when you want them. (See below under the X-Sheet section and further below in the Puppetry section.) Surfacing
You can show your model with UV Texture-Mapped surfaces, Surface Colors, Gray Shaded, or Wireframe, with full support for specularity and transparency. You can even hide surfaces to see what's going-on inside your model! X-Sheet
The X-Sheet (Exposure Sheet) has been an indispensable tool in the field of animation for over 100 years. In my experience, it is the quickest way to visually break down the complex layers of elements that go into an animated scene. With quick visual aids to help the animator identify each second, foot and half-foot of 35mm film (which has sixteen frames per foot), even animators who have never done 3D CGI animation before will feel right at home with this digital version of their old standby. Load in your Audio File, and FA automatically sets your Scene Length based upon whatever FPS you choose. (You can always make changes to Scene Length, and FPS later on). Your audio waveform is displayed on an always-visible track on the left-hand side of the X-Sheet. Click-and-drag on the waveform to hear your audio played as fast or as slow as you Scrub your mouse! Not only do you hear the individual frames of your audio played as you Scrub, you also see your animation play out, locked solidly to the audio—backwards or forwards, fast or slow!
Add Keyframes to a Dialogue Track by simply dragging and dropping Morph Icons from the Morph File onto a Dialogue Track!
Activate/Deactivate Tracks to quickly and easily see if a new idea is better than an old without loosing your old ideas! Editing Tracks in X-Sheet Want to manipulate your Keyframes without leaving the X-Sheet? No problem!
Control-clicking on a Dialogue Keyframe opens an editing window in which you can control not only the percentage that Morph is applied, but also the percentages that morph is applied to Left and Right sides of your model! (Thanks to the invisible Symmetrical Splitting mentioned earlier!) FA supports percentages below 0% and above 100%, so you can ease into your Targets, or really push a Key beyond rational for a frame (or two) for that subliminal touch that makes great dialogue animation really stand out!
Control-clicking on a Puppetry frame (even if that frame doesn't already have a Key assigned to it—great for quickly keyframing animation) opens an editing window in which you can control the positional Quad Key of that puppetry Key! (More on FA's Puppetry functions, including Quad Keys below.)
FA also supports Sliding/Cutting/Copying/Pasting/Deletion of Keys, even among different Tracks (that contain similar content)! Unlimited Tracks Something else that was important to me was the ability to layer as many tracks as I needed/wanted in order to make the animation play exactly as I needed/wanted. FA lets you add as many Layers of animation to your scene, letting you add Track after Track until what you see is exactly what you have inside your mind. And what good is unlimited Tracks unless you can interact just as quickly with your animation regardless of how many Layers/Tracks you have in your Scene? Thanks to some extreme innovation on Mac's part, FA plays 300 tracks as quickly as it plays three! Whether you're Scrubbing your Scene or playing it at speed, FA gives you instant-updates regardless of whether you have tens of tracks or hundreds! Curve Editor For those who have grown up with (or become accustomed to) 3D CGI animation, you've probably learned to think in terms of the horizontally-oriented Curve Editors for tweaking your animation to perfection. FA offers two separate Curve Editors (one for Dialogue Tracks and one for Puppeteered Tracks), so you can fine-tune your animation to your heart's content in the traditional way 3D animators have come to depend upon.
The Curve Editor for Dialogue Tracks lets you Scrub, Balance, Slide/Cut/Copy/Paste/Delete Keyframes in much the same way as the X-Sheet does, but in an interface familiar to the digital animator.
The Curve Editor for Puppeteered Tracks behaves in much the same way as the one for Dialogue, with the exception that you see separate curves for Left (green) and Right (red) components of the individual Quad Keys. Interface It's almost a given that no two artists think alike.
FA's menus are all Non-Modal, meaning you can position them, size them, hide and/or show them to create your own personal layout. FA also remembers three separate layouts so you can quickly shift between layouts optimized for different tasks within the facial animation process!
Is your character getting lost in the color of FA's Background in the Main window? No problem! FA lets you choose any color you'd like! Additionally, all eight OpenGL lights can be colored, positioned, and rotated around your model, stored with your scene so you can show your work in its best light when the director makes a surprise walkthrough.
Are the Morph Icons representing the individual shapes of your Morphs in the Morph File too small on your new 2048 px wide monitor? Just grab the corner of one of the dividers and drag to resize your Morphs to the optimal size! How 'bout that Morph in the Morph File that doesn't read well when seen from head on? Cool! Right-click and drag on any of the Icons in the Morph File to rotate them to your heart's content! FA remembers all your Morph File settings in your Scene Files! Intelligent Setup Now here's where things start getting cool. The naming of your morphs actually tells FA how to set-up a new model. FA identifies which Morphs are intended for phonetic recreation of Dialogue (and you can have as many Dialogue Morphs as you'd like) based on the Morphs' names. FA also identifies which Morphs are intended to be grouped together into Morph Palettes (see the following section on Puppeteering), if it needs to be Split into Left and Right components, and where in each Facial Palette each Morph is supposed to be!
Puppeteering One of the things that makes Yoda's performance in Star Wars' Episodes five and six, is the tactile, instant-feedback that puppetry afforded Frank Oz in creating Yoda's performance. Of equal importance was the ability for the director to request a second take, and for both director and actor to experience that take performed in real-time. With FA, Facial Palettes are used to contain certain elements of a facial gesture, such as the eyebrows arching in surprise, or scowling in anger. These elements are assigned to the upper and lower left and right corners of a Facial Palette, and when recording a Puppeteered Track, these elements are applied to your Model, based upon the position of your mouse within the Main Window.
When working with a Puppeteering Track, the Main Window shows your Capture Area, along with (fully rotatable) Iconic representations of the Morphs assigned to each corner of the current Facial Palette. Like working with paints on a painter's palette, with the quick gesture of your mouse, you blend Morphs to create unique expressions, replete with sensitivity and subtlety or explosive with broad gestures and slapstick. You instantly find that you can do a lot with the subtle arc of your mouse, easing in and out of poses, creating a subtle, asymmetrical punch to performances, doing work in seconds that would take hours or days to finesse to similar perfection (knowing that you can edit and/or re-record sections of the performance if you want). Couple this Puppeteering concept with the faster-than-real-time feedback that coding our own executable enabled, and you really see how FA can let you do in minutes what would require days to do the old fashioned way. With both Mac and me being perfectionists, we worked for several months on the mathematics that control the Blending of the different Morph Targets within the Facial Palettes. We wanted to get something as smooth as it was precise and intuitive. Eventually, we came to the realization that no one mathematical solution would work for all kinds of Morphs. With FA, you can experiment with eleven different Blending Modes, testing each with your Facial Palettes until you find the one that works best for both the Morphs and for you, the artist! You can assign specific Blending Modes to specific Facial Palettes, saving them with your Scenes and/or Default Settings! FA also has a mathematical Bumper-Zone that eases you into the most extreme of your Facial Palette's Morphs. Don't worry if you get carried away in your performance, sending your mouse winging outside the Capture Area; the Bumper Zone will keep your animation from looking like your character's expression hit a brick wall. Model Swapping FA stores its data with respect to Morph Target names, so you can swap animations (or parts of animations) between your models.
You can even swap head-models while the animation is playing!
And, if your new model may have different names for its Morphs, don't worry; FA incorporates a powerful-yet-simple Morph Remapping tool to apply your animation data to your new model's Targets. Import/Export Between Scenes When you've got a good thing going, why leave it where you found it? FA lets you Save and Load all kinds of Data between your FA Scenes! More than just animation tracks, FA lets you export and import all your preferences in whole or in part so you can quickly slide into a strange workstation and instantly feel right at home! Proven Controls It's amazing to me how little sharing of information there is among subsets of the Entertainment Industry. Other disciplines have many needs that are often very similar to our own. Often, these needs have been around for years or decades before the advent of our own, beloved, 3D CG animation. It is in our best interests to see how these other fields have met their own requirements, and to find how we can make those solutions work for us! · Playback Controls
Taking a look at the Playback Controls, you see that they look very much like a professional video editing deck's controls. Playback can be forward or backwards (also by using Jog/Shuttle style keyboard controls) at speed or at a percentage of the current frames-per-second. (From my days shooting pencil tests on an old single-frame VTR, I found this functionality to be very important for catching pops or other glitches in my animations.) Recording a Puppeteered performance can be done at the selected frame rate, or it can be done at a percentage of that speed, making it easier to anticipate for and hit all the nuances of your performance. You can choose to have your playback loop or stop when FA reaches the end of your scene. The Playback Control panel also is where you'll find the current frame indicator. And, if you've done audio recording, you'll appreciate the Punch-In/Punch-Out toggle. · Punch-In/Punch-Out
Something I loved working with as a musician and sound engineer was the Punch-In/Punch-Out functions of the professional reel-to-reel tape decks and MIDI software. Using Punch-In/Punch-Out, you could re-record just a section of a song quickly and easily. I've found this to be extremely useful when fixing a take that would be perfect... if only I hadn't missed a note, or inflection. So, having Punch-In/Punch-Out in FA, as well, was very important to me. Select a time-range in either the X-Sheet or Curve Editor, and click on the Punch-In/Punch-Out activator. Now, when you record, you re-record only this selection! · Pre-Roll Video editors and musicians alike know about Pre-Roll. In Video Editing, a Pre-Roll is where the VTRs roll the tapes backward about a second so that when the Edit In-Point is reached, the deck's heads and tape are spinning at their proper speeds. In music, Pre-Roll is (usually) a three-second span of time, punctuated by beeps and flashes on a control unit, like a band-leader counting, "One... Two... Three... Go!" before a song. With FA, you can choose to record with or without a pre-roll. Exporting Animation To Other Programs Once you have your animation perfect, you then export it to be used in other programs. FA exports to LightWave-specific Morph Mixer file format (LW versions 7-8), which allows you to modify the data within LightWave via the Morph Mixer slider-based interface. FA also offers the option to Mix-Down to a Platform Independent .MDD file, for which there are importers for all major 3D CGI platforms. (Something interesting about the .MDD file format is that it is also time-base independent; which means if you prefer animating at 24fps, and your show is being rendered at 30 fps, the frame-rate compensation is automatically handled by the .MDD importer.) Serendipity To tell the truth, I am really amazed at what FA has become. I approached Mac with specific needs and wants; he took those base ideas and pushed them well beyond what I'd even hoped for. And with FA being honed through an extremely intense production schedule (averaging 30-40 seconds of completed footage per day at the end of production), everything we could think of to make the process faster/better/easier was built into the program. The end result is a program that lets me do in about seven minutes what I would need two days to do using sliders. And the wonderful thing is that it can do the same thing for others, as well! Not only does FA make doing lip-readable dialogue synchronization and Disney feature-quality facial performances something you can bat out in a couple minutes, its ease of use means that everyone can have this kind of feature-quality work within their own projects as well! We need not be limited by the tools we've been given. In most hands-on fields of art, artists create or commission special-purpose tools all the time. "Use the right tool for the job," is a mantra I've heard hundreds of times from mentors who've instructed everything from oil painting to flute-making. The experience of working with Mac was incredibly positive. His art-form was that of computer code. He understood my needs, wants and wishes for this tool, and oftentimes had implemented requests into the program within a day. Over the course of working together, Mac also gained a solid understanding of the inner workings of feature-quality facial animation and often developed solutions from his own experience and background I would never have thought of. Would I recommend others to seek out talented, dedicated programmers with which to partner in the creation of tools that work in ways they've always thought 3D CGI should? You bet your ASCII! By not accepting the status-quo and by making available to others the tools your partnership creates, you make this field better and more enjoyable for everyone! (Artists, programmers, and audiences alike!) NOTE: Remember, Less Is More! This is especially true with dialogue animation! For Disney feature quality dialogue, use only the barest minimum number of dialogue keys needed to hold the animation in place! Timothy Albee hails from Fairbanks, Alaska. He is author of CGI Filmmaking, The Creation of Ghost Warrior, Essential LightWave 7.5, and LightWave 3D 7 Character Animation; Co-Author of Essential LightWave 8, LightWave 3D 8 Character Animation, and LightWave Getting Started Guide; and contributing author of LightWave 3D 8, 1001 Tips and Tricks. Albee's award-winning film, Kaze, Ghost Warrior was released in 2004 (http://KazeGhostWarrior.com). |
|
©DMG Publishing 2005 as an overflow from an article printed in Issue 4 of HDRI 3D, ISSN 1551-689X |
