Here comes the 3D SELFIE
The recently developed technology to produce 3D image, also helps in understanding medical conditions like autism and depression
If you love taking selfies, here comes something new to experiment with. Now you can make a 3D image from a simple image of yours. This is to note that constructing a 3D face from a two-dimensional photograph is extremely difficult. That’s because a single image makes it very hard to approximate different facial expressions across lighting conditions. But researchers at the University of Nottingham in Britain have developed a technology capable of producing 3D facial reconstruction from a single 2D image – the 3D selfie.
The new web app allows people to upload a single colour image and receive, in a few seconds, a 3D model showing the shape of their face.
Aside from the more standard applications, such as face and emotion recognition, this technology – scheduled to be presented at the International Conference on Computer Vision (ICCV) 2017 in Venice, in October – could be used to personalise computer games, improve augmented reality and let people try on online accessories such as glasses.
It could also have medical applications – such as simulating the results of plastic surgery or helping to understand medical conditions such as autism and depression.
The technique was developed using a Convolutional Neural Network (CNN) – an area of artificial intelligence (AI) which uses machine learning to give computers the ability to learn without being explicitly programmed.
The technique demonstrates some of the advances possible through deep learning - a form of machine learning that uses artificial neural networks to mimic the way the brain makes connections between pieces of information.
The Verge’s James Vincent wrote that ‘You usually need multiple pictures of the same face from different angles in order to map every contour. But, by feeding a bunch of photographs and corresponding 3-D models into a neural network, the researchers were able to teach an AI system how to quickly extrapolate the shape of a face from a single photo.’
The research team trained a CNN on a huge dataset of 2D pictures and 3D facial models. With all this information their CNN is able to reconstruct 3D facial geometry from a single 2D image. It can also take a good guess at the non-visible parts of the face. ‘Because not all elements of the face are visible in a front-facing portrait, one of the algorithm’s breakthroughs is that it can actually fabricate those hidden elements without the source material,’ says one of the researchers. For the best results, researchers recommend using a frontal photo, or as close to this as possible. You can check out an online demo of their paper, thanks to the team.
‘The main novelty is in the simplicity of our approach which bypasses the complex pipelines typically used by other techniques. We instead came up with the idea of training a big neural network on 80,000 faces to directly learn to output the 3D facial geometry from a single 2D image,’ said Georgios (Yorgos) Tzimiropoulos, Assistant Professor in the School of Computer Science.
‘Our CNN uses just a single 2D facial image, and works for arbitrary facial poses (for example, front or profile images) and facial expressions (for example, smiling),’ lead researcher Aaron Jackson said.
The task of converting 2D images into 3D has remained difficult for developers and challenged researchers in many fields. The research team would agree. They stated in their abstract, ‘3-D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty.’