Ever Wondered What Your Face Might Look Like As A 3D Model?

At the University of Nottingham and Kingston University a team of researchers have found a way to turn a 2D photo of a face into a 3D model. So if you have ever wondered what your face might look like in 3D mode, now you can…

A new algorithm has ‘learned’ how to make a 3D model from a flat image.

Thanks to the team, you can now check out an online demo of their paper. They said: “Please use a (close to) frontal image, or the face detector won’t see you (dlib).”

The researchers have a paper out on their work and the paper is on arXiv. The title is ‘Large Pose 3-D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression,’ by Aaron S. Jackson, Adrian Bulat, Vasileios Argyriou and Georgios Tzimiropoulos.

3Ders.org said that “this new Artificial Intelligence (AI) algorithm is actually pretty exciting.”

And where they merit some bragging rights is for “being able to turn a single two-dimensional image into a 3D model. This, said 3Ders.org, has remained difficult for developers and challenged researchers in many fields.

The research team would agree. They stated in their abstract, “3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty.”

How so?

Current reconstruction systems, said the authors, must address “a number of methodological challenges such as establishing dense correspondences across large facial poses, expressions, and non-uniform illumination.”

The Verge‘s James Vincent wrote that: “You usually need multiple pictures of the same face from different angles in order to map every contour. But, by feeding a bunch of photographs and corresponding 3D models into a neural network, the researchers were able to teach an AI system how to quickly extrapolate the shape of a face from a single photo.”

So how did they succeed?

3Ders.org said they trained a Convolutional Neural Network with datasets of 2D facial images – and 3D scans of the same faces. Key advantages of their CNN include its ability to work it out with just a single 2D image of a face. It does not need accurate alignment.

Co.Design said: “Because not all elements of the face are visible in a front-facing portrait, one of the algorithm’s breakthroughs is that it can actual fabricate those hidden elements without the source material.”

“The 3-D computer vision project really has to be seen to be believed,” remarked Tristan Greene in The Next Web.

Aside from project page visitors having lots of fun with this, how might their development be applied in the real world, at least in theory?

The easy scenario to guess would be for use in creating virtual reality avatars for video games.

However, as 3ders.org noted, it could also be used in the cosmetic industry to virtually test makeup.