Uncertainty aware fusion of biometric data
Biometric data may be subject to internal and external factors that complicate assessment. Internal factors include variations due to physiological changes, such as dilation or constriction of blood vessels. External factors include variations in pose, illumination, cosmetics, and so forth. One or more machine learning networks are trained to accept as input images and their modality information and provide as output parameters of conditional distributions characterizing the uncertainty due to these internal and external factors. The input images are processed by embedding modules to determine input embedding data. Based on the associated output parameters, weight values may be calculated that are associated with distances in embedding space between input embedding data from multiple modalities and other embedding data corresponding to those modalities, such as from a previously enrolled user. The weight values and distances may then be used to determine if the input images correspond to the other embedding data.
1 . A system comprising:
one or more memories, storing first computer-executable instructions; and
one or more hardware processors to execute the first computer-executable instructions to:
determine first input image data associated with a first modality, wherein the first input image data associated with the first modality is further associated with internal factors of a hand of a user;
determine first embedding data representative of the internal factors of the hand of the user as depicted in the first input image data;
determine second input image data associated with a second modality, wherein the second input image data associated with the second modality is further associated with external factors of the hand of the user;
determine second embedding data representative of the external factors of the hand of the user as depicted in the second input image data;
determine first uncertainty data indicative of uncertainties associated with the internal factors of the hand of the user;
determine second uncertainty data indicative of uncertainties associated with the external factors of the hand of the user;
determine first XOR embedding data that is representative of features that are not associated with both the first embedding data and the second embedding data;
determine third embedding data associated with the first modality, wherein the third embedding data associated with the first modality is further associated with internal factors of an enrolled hand of a corresponding user;
determine fourth embedding data associated with the second modality, wherein the fourth embedding data associated with the second modality is further associated with external factors of the enrolled hand of the corresponding user;
determine third uncertainty data indicative of uncertainties associated with the internal factors of the enrolled hand of the corresponding user;
determine fourth uncertainty data indicative of uncertainties associated with the external factors of the enrolled hand of the corresponding user;
determine second XOR embedding data that is representative of features that are not associated with both the third embedding data and the fourth embedding data;
determine a first distance in a first embedding space based on the first embedding data and the third embedding data;
determine a second distance in a second embedding space based on the second embedding data and the fourth embedding data;
determine a third distance in a third embedding space based on the first XOR embedding data and the second XOR embedding data;
determine a first weight value based on:
the first distance,
the second distance,
the third distance,
the first uncertainty data,
the second uncertainty data,
the third uncertainty data, and
the fourth uncertainty data;
determine comparison data based on:
the first distance,
the second distance,
the third distance, and
the first weight value; and
determine identification data based at least on the comparison data.
2 . The system of claim 1 , wherein one or more of the first, second, third, or fourth uncertainty data are indicative of one or more of:
uncertainty associated with physiological processes of the hand of the user as depicted in input image data, or
uncertainty associated with one or more of:
pose of the hand of the user as depicted in the input image data,
illumination of the hand of the user as depicted in the input image data, or
presence of an obscurant on the hand of the user as depicted in the input image data.
3 . The system of claim 1 , the one or more hardware processors to further execute the first computer-executable instructions to:
determine the first embedding data using a first machine learning network to process the first input image data; and
determine at least a portion of the first uncertainty data using at least a portion of the first machine learning network to process the first input image data.
4 . The system of claim 1 , the one or more hardware processors to further execute the first computer-executable instructions to:
retrieve the third embedding data and the fourth embedding data from enrolled user data; and
wherein the determination of the identification data comprises determining, based on the comparison data and the enrolled user data, an asserted identity associated with the first input image data and the second input image data.
5 . The system of claim 1 , the one or more hardware processors to further execute the first computer-executable instructions to:
determine training data comprising a plurality of entries, each entry comprising:
first modality image data,
second modality image data, and
one or more labels comprising data indicative of one or more of:
a quality label indicative of an assessed image quality,
a minutiae label indicative of friction ridge minutiae features,
an obscurant label indicative of presence of an obscuring material on at least a portion of the hand,
a visibility label indicative of an assessed visibility of features on at least a portion of the hand, or
a pose label indicative of a pose of the hand; and
train a machine learning network, using the training data, to determine at least a portion of the first uncertainty data.
6 . The system of claim 1 , the one or more hardware processors to further execute the first computer-executable instructions to:
determine first representation data using a first machine learning network to process the first input image data;
determine the first embedding data using a second machine learning network to process the first representation data;
determine second representation data using a third machine learning network to process the second input image data; and
determine the second embedding data using a fourth machine learning network to process the second representation data.
7 . The system of claim 1 , the one or more hardware processors to further execute the first computer-executable instructions to:
determine first intersection embedding data that is representative of features that are associated with both the first embedding data and the second embedding data;
determine second intersection embedding data that is representative of features that are associated with both the third embedding data and the fourth embedding data;
determine a fourth distance in a fourth embedding space based on the first intersection embedding data and the second intersection embedding data;
determine the first weight value further based on the fourth distance; and
determine the comparison data further based on the fourth distance.
8 . A computer-implemented method comprising:
determining first biometric data associated with a first modality, wherein the first biometric data associated with the first modality is further associated with internal factors of a portion of a user;
determining first embedding data representative of the internal factors of the portion of the user depicted in the first biometric data;
determining second biometric data associated with a second modality, wherein the second biometric data associated with the second modality is further associated with external factors of the portion of the user;
determining second embedding data representative of the external factors of the portion of the user depicted in the second biometric data;
determining first uncertainty data indicative of uncertainties associated with the internal factors of the portion of the user;
determining second uncertainty data indicative of uncertainties associated with the external factors of the portion of the user;
determining third embedding data associated with the first modality, wherein the third embedding data associated with the first modality is further associated with internal factors of an enrolled portion of a corresponding user;
determining fourth embedding data associated with the second modality, wherein the fourth embedding data associated with the second modality is further associated with external factors of the enrolled portion of the corresponding user;
determining third uncertainty data indicative of uncertainties associated with the internal factors of the enrolled portion of the corresponding user;
determining fourth uncertainty data indicative of uncertainties associated with the external factors of the enrolled portion of the corresponding user, wherein one or more of the first uncertainty data, the second uncertainty data, the third uncertainty data, or the fourth uncertainty data is indicative of:
uncertainty associated with physiological processes of the portion of the user depicted in the first or second biometric data;
determining a first distance in a first embedding space based on the first embedding data and the third embedding data;
determining a second distance in a second embedding space based on the second embedding data and the fourth embedding data;
determining a first weight value based on:
the first distance,
the second distance; and
one or more of:
the first uncertainty data,
the second uncertainty data,
the third uncertainty data, or
the fourth uncertainty data;
determining comparison data based on:
the first distance,
the second distance, and
the first weight value; and
determining identification data based at least on the comparison data.
9 . The computer-implemented method of claim 8 , wherein one or more of the first, second, third, or fourth uncertainty data are further indicative of one or more of:
uncertainty associated with one or more of:
pose of the portion of the user as depicted in the biometric data,
illumination of the portion of the user depicted in the biometric data, or
presence of material on the portion of the user depicted in the biometric data.
10 . The computer-implemented method of claim 8 , further comprising:
determining first XOR embedding data that is representative of features that are not associated with both the first embedding data and the second embedding data;
determining second XOR embedding data that is representative of features that are not associated with both the third embedding data and the fourth embedding data;
determining a third distance in a third embedding space based on the first XOR embedding data and the second XOR embedding data;
determining the first weight value further based on the third distance; and
determining the comparison data further based on the third distance.
11 . The computer-implemented method of claim 8 , further comprising:
determining first intersection embedding data that is representative of features that are associated with both the first embedding data and the second embedding data;
determining second intersection embedding data that is representative of features that are associated with both the third embedding data and the fourth embedding data;
determining a third distance in a third embedding space based on the first intersection embedding data and the second intersection embedding data;
determining the first weight value further based on the third distance; and
determining the comparison data further based on the third distance.
12 . The computer-implemented method of claim 8 , further comprising:
determining the first embedding data using a first machine learning network to process the first biometric data; and
determining at least a portion of the first uncertainty data using at least a portion of the first machine learning network to process the first biometric data.
13 . The computer-implemented method of claim 8 , further comprising:
retrieving the third embedding data and the fourth embedding data from enrolled user data; and
wherein the determining the identification data comprises determining, based on the comparison data and the enrolled user data, an asserted identity associated with the first biometric data and the second biometric data.
14 . The computer-implemented method of claim 8 , further comprising:
determining third biometric data associated with the first modality;
determining the third embedding data based on the third biometric data;
determining fourth biometric data associated with the second modality;
determining the fourth embedding data based on the fourth biometric data; and
wherein the first biometric data, the second biometric data, the third biometric data, and the fourth biometric data are acquired using a first device.
15 . The computer-implemented method of claim 8 , further comprising:
determining training data comprising a plurality of entries, each entry comprising:
first modality image data,
second modality image data; and
one or more labels comprising data indicative of one or more of:
a quality label indicative of an assessed image quality,
a minutiae label indicative of friction ridge minutiae features,
an obscurant label indicative of presence of an obscuring material on at least a portion of a hand,
a visibility label indicative of an assessed visibility of features on at least a portion of the hand, or
a pose label indicative of a pose of the hand; and
training a machine learning network, using the training data, to determine at least a portion of one or more of the first uncertainty data, the second uncertainty data, the third uncertainty data, or the fourth uncertainty data.
16 . The computer-implemented method of claim 8 , further comprising:
determining first representation data using a first machine learning network to process the first biometric data;
determining the first embedding data using a second machine learning network to process the first representation data;
determining second representation data using a third machine learning network to process the second biometric data; and
determining the second embedding data using a fourth machine learning network to process the second representation data.
17 . A system comprising:
one or more memories, storing first computer-executable instructions; and
one or more hardware processors to execute the first computer-executable instructions to:
determine first biometric data associated with a first modality, wherein the first biometric data associated with the first modality is further associated with internal factors of a hand of a user;
determine first embedding data representative of the internal factors of the hand of the user depicted in the first biometric data;
determine second biometric data associated with a second modality, wherein the second biometric data associated with the second modality is further associated with external factors of the hand of the user;
determine second embedding data representative of the external factors of the hand of the user depicted in the second biometric data;
determine first uncertainty data indicative of uncertainties associated with the internal factors of the hand of the user;
determine second uncertainty data indicative of uncertainties associated with the external factors of the hand of the user;
determine third embedding data associated with the first modality, wherein the third embedding data associated with the first modality is further associated with internal factors of an enrolled hand of a corresponding user;
determine fourth embedding data associated with the second modality, wherein the fourth embedding data associated with the second modality is further associated with external factors of the enrolled hand of the corresponding user;
determine third uncertainty data indicative of uncertainties associated with the internal factors of the enrolled hand of the corresponding user;
determine fourth uncertainty data indicative of uncertainties associated with the external factors of the enrolled hand of the corresponding user;
determine a first distance in a first embedding space based on the first embedding data and the third embedding data;
determine a second distance in a second embedding space based on the second embedding data and the fourth embedding data;
determine a first weight value based on:
the first distance,
the second distance; and
one or more of:
the first uncertainty data,
the second uncertainty data,
the third uncertainty data, or
the fourth uncertainty data; and
determine comparison data based on:
the first distance,
the second distance, and
the first weight value.
18 . The system of claim 17 , the one or more hardware processors to further execute the first computer-executable instructions to:
determine first XOR embedding data that is representative of features that are not associated with both the first embedding data and the second embedding data;
determine second XOR embedding data that is representative of features that are not associated with both the third embedding data and the fourth embedding data;
determine a third distance in a third embedding space based on the first XOR embedding data and the second XOR embedding data;
determine the first weight value further based on the third distance; and
determine the comparison data further based on the third distance.
19 . The system of claim 17 , the one or more hardware processors to further execute the first computer-executable instructions to:
determine the first embedding data using a first machine learning network to process the first biometric data; and
determine at least a portion of the first uncertainty data using at least a portion of the first machine learning network to process the first biometric data.
20 . The system of claim 17 , the one or more hardware processors to further execute the first computer-executable instructions to:
determine training data comprising a plurality of entries, each entry comprising:
first modality image data,
second modality image data; and
one or more labels comprising data indicative of one or more of:
a quality label indicative of an assessed image quality,
a minutiae label indicative of friction ridge minutiae features,
an obscurant label indicative of presence of an obscuring material on at least a portion of a corresponding hand,
a visibility label indicative of an assessed visibility of features on at least a portion of the corresponding hand, or
a pose label indicative of a pose of the corresponding hand; and
train a machine learning network, using the training data, to determine at least a portion of one or more of the first uncertainty data, the second uncertainty data, the third uncertainty data, or the fourth uncertainty data.