Eye texture inpainting

1. A method, comprising:

accessing, by one or more processors, a set of images as the set of images are captured at a. computing device;

determining, within a portion of each of the set of images, an area of interest containing an. eye;

segmenting, by the one or more processors, a sclera area within the area of interest, the sclera area corresponding to a sclera of the eye;

determining a color distribution for the sclera area;

performing an initial match of the sclera area to a subset of a set of sclera templates based on a prevailing color of the color distribution of the sclera area, the set of sclera templates generated prior to receiving the set of images; and

selecting a sclera template from the subset of the sclera templates that matches or approximates one or more color variations or patterns within one or more color variations of the sclera area;

modifying a portion of the selected sclera template; and

generating, by the one or more processors, a texture for the eye based on the selected sclera template including the modified portion of the selected sclera template.

2. The method of claim 1 , further comprising:

generating a circular area for an iris within the area of interest, the generating comprising:

determining an iris measurement comprising a diameter or circumference of the iris including a portion of the iris obscured by one or more eyelids; and

generating a circular shape with dimensions or measurements matching or approximating the iris measurement;

identifying an unknown area of the iris, the unknown area corresponding to a portion of the iris obstructed by an eyelid;

determining one or more color patches within the circular area, the one or more color patches corresponding to a colored portion of the iris; and

translating the color patches to the unknown area.

3. The method of claim 2 , wherein generating the circular area further comprises:

identifying at least a portion of an edge of the iris within the area of interest;

determining the circular shape, sized based on the portion of the edge of the iris;

aligning the circular shape with the portion of the edge of the iris; and

resizing a size, shape or dimension of the circular shape within the area of interest such that at least a portion of the circular shape intersects or touches an iris area.

4. The method of claim 2 further comprising:

determining the unknown area is filled with the translated one or more color patches; and

reverting the area of interest to an original color coordinate system for the image prior to translating the color patches.

5. The method of claim 1 , wherein segmenting the sclera area further comprises:

determining a portion of the area of interest corresponding to the sclera.

6. The method of claim 1 , wherein generating the texture for the eye further comprises:

extracting an iris area from the area of interest, the iris area corresponding to at least one iris depicted in at least a portion of the set of images;

generating an iris texture for the iris area, the texture based on at least one or more color patches selected from the iris area;

selecting a sclera texture from a set of predetermined sclera textures, the sclera texture selected based on a color distribution of the sclera area; and

blending an edge of the iris texture with a portion of the sclera texture proximate to the iris area.

7. The method of claim 1 , further comprising:

identifying a pupil location within the area of interest, the pupil location determined based on an iris area and the sclera area;

determining a first point of focus for eye, the first point of focus based on the pupil location; and

modifying a portion of the texture for the eye to change the first point of focus to a second point of focus, the second point of focus directed to a position different than that of the first point of focus.

8. The method of claim 1 , further comprising:

determining a set of facial landmarks for a face depicted within the set of images;

determining a face model for the face depicted within the set of images, the face model having a three-dimensional representation of the eyes and aligned with at least a portion of the set of facial landmarks for the face;

identifying a pupil location within the area of interest, the pupil location determined based on an iris area and the sclera area; and

tracking a point of focus of the eye based on the pupil location and the face model.

9. A system, comprising:

one or more processors; and

non-transitory processor-readable storage medium storing processor-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

accessing a set of images as the set of images are captured at a computing device;

determining, within a portion of each of the set of images, an area of interest containing an eye;

segmenting a sclera area within the area of interest, the sclera area corresponding to a sclera of the eye;

determining a color distribution for the sclera area;

performing an initial match of the sclera area to a subset of a set of sclera templates based on, a prevailing color of the color distribution of the sclera area, the set of sclera templates generated prior to receiving the set of images; and

selecting a sclera template from the subset of the sclera templates that matches or approximates one or more color variations or patterns within one or more color variations of the sclera area;

modifying a portion of the selected sclera template; and

generating, by the one or more processors, a texture for the eye based on the selected sclera template including the modified portion of the selected sclera template.

10. The system of claim 9 , wherein the operations further comprise:

generating a circular area for an iris within the area of interest, the generating comprising:

determining an iris measurement comprising a diameter or circumference of the iris including a portion of the iris obscured by one or more eyelids; and

generating a circular shape with dimensions or measurements matching or approximating the iris measurement;

identifying an unknown area of the iris, the unknown area corresponding to a portion of the iris obstructed by an eyelid;

determining one or more color patches within the circular area, the one or more color patches corresponding to a colored portion of the iris; and

translating the color patches to the unknown area.

11. The system of claim 10 , wherein generating the circular area further comprises:

identifying at least a portion of an edge of the iris within the area of interest;

determining the circular shape, sized based on the portion of the edge of the iris;

aligning the circular shape with the portion of the edge of the iris; and

resizing a size, shape or dimension of the circular shape within the area of interest such that at least a portion of the circular shape intersects or touches an iris area.

12. The system of claim 9 , wherein generating the texture for the eye further comprises:

extracting an iris area from the area of interest. the iris area corresponding to at least one iris depicted in at least a portion of the set of images;

generating an iris texture for the iris area, the texture based on at least one or more color patches selected from the iris area;

selecting a sclera texture from a set of predetermined sclera textures, the sclera texture selected based on a color distribution of the sclera area; and

blending an edge of the iris texture with a portion of the sclera texture proximate to the iris area.

13. The system of claim 9 , wherein the operations further comprise:

identifying a pupil location within the area of interest, the pupil location determined based on an iris area and the sclera area;

determining a first point of focus for eye, the first point of focus based on the pupil location; and

modifying a portion of the texture for the eye to change the first point of focus to a second point of focus, the second point of focus directed to a position different than that of the first point of focus.

14. The system of claim 9 , wherein the operations further comprise:

determining a set of facial landmarks for a face depicted within the set of images;

determining a face model for the face depicted within the set of images, the face model having a three-dimensional representation of the eyes and aligned with at least a portion of the set of facial landmarks for the face;

identifying a pupil location within the area of interest, the pupil location determined based on an iris area and the sclera area; and

tracking a point of focus of the eye based on the pupil location and the face model.

15. A non-transitory processor-readable storage medium storing processor executable instructions that, when executed by a processor of a machine, cause the machine to perform operations comprising:

accessing a set of images as the set of images are captured at a computing device;

determining, within a portion of each of the set of images, an area of interest containing an eye;

segmented a sclera area within the area of interest, the sclera area corresponding to a sclera of the eye depicted in at least a portion of the set of images;

determining a color distribution for the sclera area;

performing an initial match of the sclera area to a subset of a set of sclera templates based on a prevailing color of the color distribution of the sclera area, the set of sclera templates generated prior to receiving the set of images; and

selecting a sclera template from the subset of the sclera templates that matches or approximates one or more color variations or patterns within one or more color variations of the sclera area;

modifying a portion of the selected sclera template; and

generating, by the one or more processors, a texture for the eye based on the selected sclera template including the modified portion of the selected sclera template.

16. The non-transitory processor-readable storage medium of claim 15 , wherein the operations further comprise:

generating a circular area for an iris within the area of interest, the generating comprising:

determining an iris measurement comprising a diameter or circumference of the iris including a portion of the iris obscured by one or more eyelids; and

generating a circular shape with dimensions or measurements matching or approximating the iris measurement;

identifying an unknown area of the iris, the unknown area corresponding to a portion of the iris obstructed by an eyelid;

determining one or more color patches within the circular area, the one or more color patches corresponding to a colored portion of the iris; and

translating the color patches to the unknown area.

17. The non-transitory processor-readable storage medium of claim 16 , wherein generating the circular area further comprises:

identifying at least a portion of an edge of the iris within the area of interest;

determining the circular shape, sized based on the portion of the edge of the iris;

aligning the circular shape with the portion of the edge of the iris; and

resizing a size, shape or dimension of the circular shape within the area of interest such that at least a portion of the circular shape intersects or touches an iris area.

18. The method of claim 1 , further comprising:

determining an iris measurement comprising a diameter or circumference of the iris including a portion of the iris obscured by one or more eyelids; and

generating a circular shape with dimensions or measurements matching or approximating the iris measurement.

19. The method of claim 18 further comprising resizing a size, shape or dimension of the circular shape within the area of interest such that at least a portion of the circular shape intersects or touches an iris area.

20. The non-transitory processor-readable storage medium of claim 15 , wherein the operations further comprise:

extracting an iris area from the area of interest, the iris area corresponding to at least one iris depicted in at least a portion of the set of images;

determining a set of facial landmarks for a face depicted within the set of images;

determining a face model for the face depicted within the set of images, the face model having a three-dimensional representation of the eyes and aligned with at least a portion of the set of facial landmarks for the face;

identifying a pupil location within the area of interest, the pupil location determined based on the iris area and the sclera area; and

tracking a point of focus of the eye based on the pupil location and the face model.