The Hands-Free Videophone captures the user’s face with three cameras in each of the left and right sides of the frames. The video sent to the other person is created by combining the pictures with a pre-rendered 3D model of the users face.
“Each camera has 720p resolution, and a fish-eye lens, with a 180-degree field of view. This is the High Definition picture currently being captured in real time. If you look at the face, you can see it’s really distorted, because the fish-eye lens is so close. The distortion is compensated, and the picture is combined with a 3D model of the person in the computer. Currently, priority is given to the part around the eyes. As you can see when the man closes his eyes, the eyelids and the corners of the eyes appear quite realistic. Such a level of realism is hard to achieve with models like CG-based avatars, where parts are overlaid on the face.”