System for enabling eye contact in electronic images

1. A computer program product comprising one or more non-transient computer-readable mediums having encoded thereon executable instructions that, when executed by at least one processor, cause the at least one processor to carry out a process of producing a face image, the process comprising:

causing capture of an electronic image of the user with a camera;

creating a three-dimensional mesh having a plurality of points from at least a portion of the electronic image that includes an eye area of the user, each point in the three-dimensional mesh being associated with color or grayscale information in the electronic image;

identifying a viewing angle of the user by calculating, at each point in the three dimensional mesh, a plurality of surface normals orthogonal to a plane that is tangent to a surface of the eye area of the user, and calculating a mean surface normal from the plurality of surface normals, wherein the viewing angle of the user is the mean surface normal;

estimating an angle of rotation for gaze correction;

rotating the three-dimensional mesh, based on the angle of rotation, to produce a rotated image of at least the eye area of the user, the rotated image including color or grayscale information associated with each point of the rotated three-dimensional mesh; and

updating the electronic image of the user with the color or grayscale information in the rotated image.

2. The computer program product of claim 1 , wherein causing capture of the electronic image of the user includes capturing the electronic image with two or more calibrated cameras.

3. The computer program product of claim 2 , wherein creating the three-dimensional mesh includes creating the three-dimensional mesh from depth information obtained through rectifying the electronic image as captured by the two or more calibrated cameras and producing a disparity map.

4. The computer program product of claim 1 , wherein rotating the three-dimensional mesh includes rotating the three-dimensional mesh to minimize the viewing angle with respect to a line of sight between the camera and an eye of the user.

5. The computer program product of claim 1 , wherein the eye area is identified from a disparity map of the electronic image.

6. The computer program product of claim 1 , wherein rotating the three-dimensional mesh includes adjusting a tilt angle of the three-dimensional mesh to produce a rotated image of the user that provides apparent eye contact.

7. The computer program product of claim 6 , wherein rotating the three-dimensional mesh further includes adjusting at least one of a pan angle and rotation angle of the three-dimensional mesh to produce a rotated image of the user.

8. The computer program product of claim 1 , further comprising:

identifying whether the user is looking to a lateral side with respect to the camera and foregoing updates to the electronic image when the user is looking to the lateral side.

9. A mobile computing device comprising the computer program product of claim 1 and the camera of claim 1 .

10. A system, comprising:

a camera;

one or more processors; and

a memory having encoded thereon executable instructions that, when executed by the one or more processors, cause the one or more processors to carry out a process of enabling eye contact in imaging systems, the process including:

capturing an electronic image of the user with the camera;

creating a three-dimensional mesh having a plurality of points from at least a portion of the electronic image that includes an eye area of the user, each point in the three-dimensional mesh being associated with color or grayscale information in the electronic image;

identifying a viewing angle of the user by calculating, at each point in the three dimensional mesh, a plurality of surface normals orthogonal to a plane that is tangent to a surface of the eye area of the user, and calculating a mean surface normal from the plurality of surface normals, wherein the viewing angle of the user is the mean surface normal;

estimating an angle of rotation for gaze correction;

rotating the three-dimensional mesh, based on the angle of rotation, to produce a rotated image of at least the eye area of the user, the rotated image including color or grayscale information associated with each point of the rotated three-dimensional mesh; and

updating the electronic image of the user with the color or grayscale information in the rotated image.

11. The system of claim 10 , wherein capturing the electronic image includes capturing the electronic image and depth information with a camera and a depth sensor, respectively.

12. The system of claim 11 , wherein creating the three-dimensional mesh includes creating the three-dimensional mesh from depth information captured by the depth sensor.

13. The system of claim 12 , wherein creating the three-dimensional mesh includes creating a point cloud of at least a portion of the eye area.

14. The system of claim 10 , wherein creating the three-dimensional mesh from at least a portion of the electronic image that includes a face and torso area of the user.

15. The system of claim 14 , wherein rotating the three-dimensional mesh to produce the rotated image includes rotating the face and torso area of the user.

16. The system of claim 10 , wherein capturing the electronic image includes capturing an electronic image of a plurality of users.

17. The system of claim 16 , further comprising:

creating a three-dimensional mesh from at least a portion of each of the plurality of users;

identifying a viewing angle for each of the plurality of users; and

updating the electronic image independently for each of the plurality of users.

18. The system of claim 10 , wherein the system is a mobile computing device.

19. The system of claim 18 , wherein the mobile computing device is a smartphone, a tablet computer, or a laptop computer.

20. A method of enabling eye contact in imaging systems, the method comprising:

capturing an electronic image of the user with a camera;

creating a three-dimensional mesh having a plurality of points from at least a portion of the electronic image that includes an eye area of the user, each point in the three-dimensional mesh being associated with color or grayscale information in the electronic image;

identifying a viewing angle of the user by calculating, at each point in the three dimensional mesh, a plurality of surface normals orthogonal to a plane that is tangent to a surface of the eye area of the user, and calculating a mean surface normal from the plurality of surface normals, wherein the viewing angle of the user is the mean surface normal;

estimating an angle of rotation for gaze correction;

rotating the three-dimensional mesh, based on the angle of rotation, to produce a rotated image of at least the eye area of the user, the rotated image including color or grayscale information associated with each point of the rotated three-dimensional mesh; and

updating the electronic image of the user with the color or grayscale information in the rotated image.

21. The method of claim 20 , wherein rotating the three-dimensional mesh includes rotating the three-dimensional mesh to minimize the viewing angle with respect to a line of sight between the camera and an eye of the user.

22. The method of claim 20 , wherein capturing and updating the electronic image includes capturing and updating an electronic video image.

23. The method of claim 20 , wherein capturing and updating the electronic image includes capturing and updating a still image.