Deep neural network for iris identification
Systems and methods for iris authentication are disclosed. In one aspect, a deep neural network (DNN) with a triplet network architecture can be trained to learn an embedding (e.g., another DNN) that maps from the higher dimensional eye image space to a lower dimensional embedding space. The DNN can be trained with segmented iris images or images of the periocular region of the eye (including the eye and portions around the eye such as eyelids, eyebrows, eyelashes, and skin surrounding the eye). With the triplet network architecture, an embedding space representation (ESR) of a person's eye image can be closer to the ESRs of the person's other eye images than it is to the ESR of another person's eye image. In another aspect, to authenticate a user as an authorized user, an ESR of the user's eye image can be sufficiently close to an ESR of the authorized user's eye image.
1. A system comprising:
an authentication training system configured to execute one or more deep neural networks to determine an embedding configured to determine embedding space representations of eye images using a deep neural network with a triplet network architecture, wherein the authentication training system is configured to:
access an anchor eye image of a user, a positive eye image of the user, and a negative eye image of another user, wherein the user is an authorized user;
map, using the one or more deep neural networks, the anchor eye image to a first image embedding space to generate an anchor embedding space representation;
map, using the one or more deep neural networks, the positive eye image to the first image embedding space to generate a positive embedding space representation;
map, using the one or more deep neural networks, the negative eye image to the first image embedding space to generate a negative embedding space representation;
update the one or more deep neural networks to:
reduce a distance in the first image embedding space between the positive embedding space representation and the anchor embedding space representation; or
increase a distance in the first image embedding space between the negative embedding space representation and one or more of the positive and anchor embedding space representations;
generate the embedding based on the updated one or more deep neural networks; and
generate at least one classifier usable to determine a likelihood that embedding space representations of images are associated with the user; and
a wearable display system comprising:
a display;
an image capture device configured to capture eye images of a wearer, wherein the wearable display system is configured to:
receive a first eye image from the image capture device;
receive, from the authentication training system, the embedding and the at least one classifier;
process the first eye image to generate a representation of the first eye image in polar coordinates;
process the representation of the first eye image using the embedding to generate a first embedding space representation of the first eye image, wherein the first embedding space representation is an n-dimensional vector, and
wherein a majority of elements of the first embedding space representation are statistically independent;
process the first embedding space representation using the at least one classifier to calculate a likelihood score that the first eye image is associated with an authorized user;
grant or deny the wearer access to the wearable display system based on the likelihood score;
transmit the first eye image to the authentication training system; and
receive an updated embedding determined by the authentication training system using the first eye image.
2. The system of claim 1 ,
wherein the one or more deep neural networks comprise a plurality of layers, and
wherein the plurality of layers comprises a pooling layer, a brightness normalization layer, a convolutional layer, an inception-like layer, a rectified linear layer, a softsign layer, or any combination thereof.
3. The system of claim 2 , wherein the brightness normalization layer comprises a local contrast normalization layer, a local response normalization layer, or a combination thereof.
4. The system of claim 1 , wherein the first embedding space representation has unit length.
5. The system of claim 1 , wherein the at least one classifier determines the likelihood score based on a Euclidian distance.
6. The system of claim 1 , wherein the at least one classifier is a binary classifier, a logistic regression classifier, a support vector machine classifier, a Bayesian classifier, a softmax classifier, or any combination thereof.
7. The system of claim 1 ,
wherein the wearable display system is configured to: segment the first eye image to generate a second eye image, and
wherein to process the first eye image, the wearable display system is configured to: process the second eye image using the embedding to generate the first embedding space representation.
8. The system of claim 1 , wherein the one or more deep neural networks are configured to learn the embedding from a set of biometric information, wherein the set of biometric information comprises at least one of skin tone, skin texture, fingerprints, or voice.
9. The system of claim 1 , wherein the one or more deep neural networks are is further configured to learn a co-embedding for secondary biometric information and wherein the wearable display system is further programmed to:
process biometric data using the co-embedding to generate a co-embedding space representation based on the biometric data; and
process the co-embedding space representation using the at least one classifier to calculate the likelihood score that the first eye image of the eye is an image of an eye of an authorized user.
10. A head mounted display system comprising:
a display;
an image capture device configured to capture images of a first and second eye of a user;
non-transitory memory configured to store executable instructions; and
a hardware processor in communication with the display, the image capture device, and the non-transitory memory, the hardware processor programmed by the executable instructions to:
receive a first image of the first eye and a second image of the second eye;
receive at least one embedding and at least one classifier from an authentication training system configured to execute one or more deep neural networks to learn an embedding space for eye images of a plurality of persons, wherein the at least one embedding comprises a mapping from an image space to a learned embedding space;
process the first image of the first eye and the second image of the second eye to generate a representation of the first image of the first eye in polar coordinates and a representation of the second image of the second eye in polar coordinates;
process the representation of the first image of the first eye in polar coordinates and the representation of the second image of the second eye in polar coordinates using the at least one embedding to generate a first embedding space representation and a second embedding space representation, wherein the first embedding space representation or the second embedding space representation is an n-dimensional vector, and wherein a majority of elements of the first embedding space representation or the second embedding space representation are statistically independent;
process the first embedding space representation and the second embedding space representation using the at least one classifier to generate a likelihood score that an image of the eye is an image of an authorized user's eye, wherein the image of the eye comprises the first image of the first eye or the second image of the second eye;
transmit, based on the likelihood score, the image of the eye to the authentication training system; and
receive an updated embedding from the authentication training system.
11. The head mounted display system of claim 10 , wherein the one or more deep neural networks is trained using a triplet network.
12. The head mounted display system of claim 11 ,
wherein the triplet network is configured to learn the one or more deep neural networks from eye images of a plurality of persons, and
wherein a distance in an embedding space representation based on eye images from the first eye of an individual person is smaller than a distance in an embedding space representation based on eye images from different persons and smaller than a distance in an embedding space representation based on eye images from the second eye of the individual person.
13. The head mounted display system of claim 10 , wherein the hardware processor is programmed by the executable instructions to:
grant or deny the user access to the head mounted display system based on the likelihood score.
14. The head mounted display system of claim 10 ,
wherein the hardware processor is programmed by the executable instructions to: segment the first image of the first eye to generate a third image of an iris of the first eye, and
wherein to process the first image of the first eye, the hardware processor is programmed by the executable instructions to: process the third image of the iris of the first eye using the one or more deep neural networks to generate the first embedding space representation.
15. The head mounted display system of claim 10 , wherein the first image of the first eye comprises mostly of the iris and the retina of the first eye.
16. The head mounted display system of claim 10 , wherein the first image of the first eye comprises mostly of the retina of the first eye.
17. The head mounted display system of claim 10 , wherein the one or more deep neural networks are is configured to generate an embedding space representation based on a set of biometric information, wherein the set of biometric information comprises at least one of skin tone, skin texture, fingerprints, or voice.
18. The head mounted display system of claim 10 , wherein the hardware processor is further programmed to:
process biometric data to generate a co-embedding space representation; and
process the co-embedding space representation using the at least one classifier to calculate the likelihood score that the first image of the first eye or the second image of the second eye is an image of an authorized user's eye.
19. The head mounted display system of claim 10 , wherein the hardware processor is further programmed to:
process biometric data to generate a co-embedding space representation; and
process the co-embedding space representation using the at least one classifier to calculate the likelihood score.
20. A wearable device comprising:
a display;
an image capture device configured to capture a first image of an eye of a user;
non-transitory memory configured to store executable instructions; and
a hardware processor in communication with the display, the image capture device, and the non-transitory memory, the hardware processor programmed by the executable instructions to:
receive the first image of the eye from the image capture device;
receive an embedding from an authentication training system configured to execute one or more deep neural networks to learn an embedding space for eye images of a plurality of persons, wherein the embedding is useable to map images from an image space to an embedding space;
process the first image of the eye to generate a representation of the first image of the eye in polar coordinates;
process the representation of the first image of the eye
from an image space to an embedding space;
process the first image of the eye to generate a representation of the first image of the eye in polar coordinates;
process the representation of the first image of the eye using the embedding to generate an embedding space representation of the first image of the eye;
receive a classifier from the authentication training system, the classifier useable to determine a distance between the embedding space representation of the first image of the eye and one or more embedding space representations of eye images of authorized users;
process the embedding space representation of the first eye using the classifier to calculate a likelihood score that the first image of the eye is associated with an authorized user, wherein the embedding space representation of the first eye is an n-dimensional vector, and wherein the majority of elements of the embedding space representation of the first eye are statistically independent;
determine, based on the likelihood score, an authentication level of the user;
transmit the first image of the eye to the authentication training system; and
receive an updated embedding determined by the authentication training system using the first image of the eye.
21. The wearable device of claim 20 , wherein the one or more deep neural networks is trained using a triplet network.
22. The wearable device of claim 20 , wherein the one or more deep neural networks are configured to learn the embedding from a set of biometric information, wherein the set of biometric information comprises at least one of skin tone, skin texture, fingerprints, or voice.