A new era in selfies could begin thanks to a technology developed by computer scientists at the University of Nottingham and the University of Kingston. 3D facial reconstruction is now possible from a single image: the 3D selfie.
Through a web application, anyone can upload a single color image and receive, in a few seconds, a 3D model of it. More than 400,000 users have already taken the test.
The technique, while far from perfect, is a breakthrough that computer scientists have been looking for for years. What does it consist of? The new technology is based on a convolutional neural network, that is, an area of Artificial Intelligence that uses machine learning to provide computers with the ability to learn without being explicitly programmed for it.
The research team, supervised by Dr. Yorgos Tzimiropoulos, developed this convolutional neural network based on a huge dataset of 2D images and 3D facial models. With all this information, his neural network was able to reconstruct the 3D facial geometry from a single 2D image. Furthermore, he was able to reconstruct the non-visible parts of the face by making a series of guesses.
“The main novelty is in the simplicity of our approach, unlike other techniques,” acknowledges the director of the research, Dr. Tzimiropoulos.
The systems available to date required multiple facial images and faced various challenges, such as the exact matching of facial poses, expressions, and non-uniform lighting.
“Our neural network uses only a 2D facial image, and works with arbitrary facial poses (for example, frontal or profile images) and facial expressions (for example, smiling),” explains Aaron Jackson, another of the researchers.
The new technique is a showcase of advances in deep learning , a form of machine learning that uses artificial neural networks (AI) to mimic the way the brain makes connections between pieces of information.
Multiple applications
In addition to more obvious applications, such as face and emotion recognition , this technology could be used to personalize games, enhance virtual reality, and allow consumers to test accessories online, such as sunglasses.
The new 3D facial reconstruction technique could even have medical applications; for example, simulating the results of plastic surgery, or it could help to understand certain medical conditions such as autism or depression.
