In the hit 1997 movie Face/Off, John Travolta and Nicolas Cage play guys who swap their faces for some presumably whoa! reason that I can't actually remember. Anyway, once the facial transfer takes place, bucket loads of mistaken-identity-related violence predictably ensue.
Nobody really talks about that movie anymore, but what's amazing is how close scientists are getting to pulling off the same trick with new technology. The video above demonstrates what the researchers call "Real-time Expression Transfer for Facial Reenactment". It's a bit like Face/Off, except instead of swapping whole faces, the technology transfers facial expressions from one face to another.
In the video you can see the actor on the right pull a bunch of exaggerated facial expressions like crazy smiles, raised eyebrows, and frowns. All of those gestures are mapped in real time onto the face of the individual sitting on the left.
At first you might not notice this, but it becomes quickly apparent once you only pay attention to the person and the screen on the left. You'll note that the target actor in the flesh keeps their face calm, but their onscreen avatar is acting like a virtual lunatic, mugging a parade of non-sensical facial expressions that perfectly match what the source actor (and their accompanying onscreen display) on the right are doing.
What's so impressive is the facial gestures aren't just mapped onto a still, cardboard cutout. As the target actor moves his head around or shifts in his chair, the source actor's facial expressions keep in check with the angle and perspective to create a flowing virtual animation – a hybrid that effectively combines one person's body with another's emotional state (at least to the extent that emotions are communicated by facial expressions).
The effect is achieved by the software effectively 'mapping' the two actors' faces and then laying elements of the source actor's face over the target individual's face with a kind of CGI skin wrapper, then enhancing the realism with various artificially animated CGI renders (like teeth) to fill out the illusion.
Much as is the case with some CGI movies and animations, the biggest giveaway that the virtual actor you're watching on the screen isn't real is in the eyes. If you look closely at the target actor reenactment in some of the demos, you can see the eyes have a kind of zombie-like, deadened stare to them that kills the effect somewhat.
But it's still a truly mind-blowing advance for video animation, and we can't wait to see what some of the practical applications of this technique could be.
The research is published on the Stanford University website.