The news is full of articles about an innovative tool that uses Artificial Intelligence to change how old actors look in movies. This is not new – actors’ apparent ages in movies have been manipulated in different ways for a long time. What is new is the believability and ease of use of this method, called FRAN (Face Re-Aging Network), developed by UW–Madison Computer Sciences Professor Eftychios Sifakis and his collaborators at Disney Research.
Until now, making actors look older or younger has been a laborious, time-consuming process. There are two methods studios use: first, a method in which a full three-dimensional model of the actor is created. This model is then matched to the performance of the live actor. This process can create high quality results even in close-up shots but is very time consuming as the model needs to be endowed with intricate controls – almost like a puppet – and those must be tediously adjusted to match the actor’s performance.
The other method uses two-dimensional images and manipulates them frame by frame with tools similar to airbrushing facial wrinkles, folds, and blemishes in Photoshop. This is also very time consuming as it needs to happen image-by-image and in a way that keeps edits consistent from one frame to the next.
To create FRAN, the new and improved method of making actors look older or younger, Sifakis and the Disney Research group took an existing neural network-based tool, StyleGAN2 (famously used by thispersondoesnotexist.com), which can create believable human faces, and leveraged earlier research that can re-age forward and backward specifically the faces created by this neural network. StyleGAN2 contains many human faces but not every possible one. However, it provides an adequately rich sampling of such identities to create a database that exemplifies the entire spectrum of human faces very well. Using this database of identities, each paired with a progression of ages from 18 to 85 years old, FRAN learns the rules by which any given identity – even those beyond the ones that StyleGAN2 can generate – can be consistently adjusted to look younger or older while keeping the identity of the person plausibly unchanged. Sifakis emphasized that “the ability of FRAN to adjust an actor’s age without altering their apparent identity or compromising visual details is an important and distinctive feature of this work.”
Disney Research YouTube video about FRAN
Sifakis has been working on computer graphics and specifically applications to animations of virtual humans for nearly two decades. “Part of my research journey has been about generating virtual characters using simulation, driving facial expressions by recreating the action of muscles, flesh and skin based on the physical laws that govern such materials” said Sifakis. “In more recent years, artificial intelligence has allowed animation researchers to leverage large datasets of facial performances as a complementary process to simulation in crafting and editing animations of digital actors.” This collaborative project is a perfect example of such infusion of Artificial Intelligence and Machine Learning techniques as valuable new tools for bringing digital humans to life. The research that led to FRAN is being presented in a paper written by Sifakis and his team at the Siggraph Asia conference in Daegu, South Korea this week.
Eftychios Sifakis is an Associate Professor and Associate Chair of the UW–Madison Computer Sciences Department, which is part of the School of Computer, Data & Information Sciences. He earned his PhD from Stanford University. His current research revolves around the areas of computer graphics, physics-based modeling/simulation and scientific computing. He is particularly interested in the development of algorithms and numerical techniques that can facilitate the efficient and accurate simulation of parts of the human body, with focus on applications in biomechanics and virtual surgery.
The UW-Madison Department of Computer Sciences is one of the oldest and most respected CS departments in the US, with educational offerings that include undergraduate and MS/PhD degrees and professional graduate programs. We provide an unbeatable learning environment for students at all levels, in all areas of computer science, and innovative discoveries by our researchers and through collaborations with others have seeded many high-tech start-ups.
The School of Computer, Data & Information Sciences (CDIS) brings together the top ranked departments of Computer Sciences, Statistics, and the Information School. Supporting Sifakis’s research is only one of the ways CDIS advances research and discovery, magnifying the power of medicine, engineering, agriculture, business and more. In 2025, CDIS will open its new building that will create a new center of activity for UW, fostering research, collaboration and learning opportunities.