Face2Face: Real-Time Facial Reenactment | Two Minute Papers #63
Face2Face: Real-Time Facial Reenactment | Two Minute Papers #63

Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér. There was a previous episode on a technique
where the inputs were a source video of ourselves, and a target actor. And the output was a video
of this target actor with our facial gestures. With such an algorithm, one can edit pre-recorded
videos in real time, and the current version only needs a consumer webcam to do that. This new version addresses two major shortcomings:
One: the previous work relied on depth information, which means that we needed to know how far
different parts of the image were from the camera. This newer version only relies on
color information and does not need anything beyond that. Whoa!
Two: Previous techniques often resorted to copying the footage from the mouth and adding
synthetic proxies for teeth. Not anymore with this one! I tip my hat to the authors, who came up with
a vastly improved version of their previous method so quickly, and it is probably needless
to say that the ramifications of such an existing technique are far reaching, and are hopefully
pointed in a positive direction. However, we should bear in mind that from
now on, we may be one step closer to an era where a video of something happening won’t
be taken as proper evidence. I wonder how this will affect legal decision-making in
the future. Thanks for watching, and for your generous
support, and I’ll see you next time!

22 thoughts on “Face2Face: Real-Time Facial Reenactment | Two Minute Papers #63”

  1. Pedro Afonso Oitaven de Sousa says:


  2. StormCrow says:

    Videos of "Hello, I'm an African prince and I would like to give you all my money" incoming.

  3. Joseph Gericke says:

    the only thing i can think of is what awesome parodys of movies u can make with this

  4. kkass says:

    This reminds me of a process described in Henry Kuttner's "Fury", where a video falsified in such a fashion was used to defame a public figure. Feels surreal how this science fiction technology is no longer fiction.

  5. Neoshaman Fulgurant says:

    More impressed by the idea of High Quality™ markerless facial motion capture using commodity camera.

  6. Kane says:

    wow, super impressive results

  7. Russell Cox says:

    You bring up a good point about the legal ramifications regarding evidence. What about impersonating someone's likeness?

  8. David McSween says:

    Its been quite some time now since people have considered video to be infallible. Modern CGI can make anything seem real. However it's the context and details that separate forgery from reality. Perhaps a form of Morellian Analysis and demonstration of the "Chain of Custody" that will show veracity of a document.

  9. David McSween says:

    I wonder if the recent deep thought painting tool could be turned to discover forgeries instead of making them?

  10. Krishna Penukonda says:

    Is there a place where we can get the code to play around with it?

  11. UnknownSentinel says:

    Is the source code for the demo accesible somewhere?

  12. ProGamerGov says:

    So has the code been released? Or is someone working on implementation of this? Can't find a download link of the software anywhere.

  13. Geo Synchronous says:

    It will only be used to deceive people. This planet is getting way to far out of control.

  14. Andrei Brănescu says:

    Wait until we'll have the same feature for voice. You'll be able to make Trump say anything you want. 🙂

  15. Death Barrier says:

    So you just use RGB data, how about how do you transfer and modify the mesh from the actor to the target without having the mesh of the actor overlapped on top of the target?

  16. Hans Sprungfeld says:

    How can this possibly be used in a benign way?!?

  17. HotchiwawaChronicles says:

    Don't believe pictures, don't believe video anymore.

  18. Tymski says:

    How can I get this going on my machine?

  19. Trush Tucker says:

    love this.i have more opportunities to make movies or series

  20. Ha Vu Tran says:

    Now films can be lip-synced to different languages 🙂 Towards a better poly-lingual hollywood

  21. M. Manel says:

    where we can download ???

  22. Union Village Media says:

    but the real question is… when is it going to be in the app store!? LOL

Leave a Reply

Your email address will not be published. Required fields are marked *