Identifying partially covered objects utilizing machine learning
Techniques are disclosed for determining the presence of a particular person based on facial characteristics. For example, a device may include a first image in a reference set of images based on determining that a face shown in the first image is not covered by a face covering. A trained model of the device may determine a first set of characteristics from the first image, whereby the trained model is trained utilizing simulated face coverings to match a partially covered face of a particular person with a non-covered face of the particular person. The device may also determine a second set of characteristics associated with a face of a second person based on a second image. The trained model may then determine a score corresponding to a level of similarity between both sets of characteristics, and then determine whether the first person is the second person based on the score.
1 . A computer-implemented method, comprising:
accessing, by a first device associated with a first user, a first image locally stored on the first device associated with the first user, the first image comprising a portion of a face of a first person, the first image being included within a set of reference images for the face of the first person based at least in part on a determination that the portion of the face of the first person shown in the first image is not covered by a face covering, and the set of reference images respectively showing non-covered portions of the face of the first person;
receiving, by the first device from a second device that is a different device from the first device and the second device comprises a camera, a second image for comparison against the first image of the first person that is not covered by the face covering, the second image used by a training module of the first device to determine whether the first image of the first person and the second image of a second person correspond to a same person, the second image comprising a portion of a face of the second person, the portion comprising a face covering that partially covers the face of the second person; and
determining, using the training module of the first device, whether the first image of the first person and the second image of the second person received from the second device that is the different device from the first device corresponds to the same person, the training module being trained to associate a full image of a partially covered face of a particular person with a full image of a non-covered face of the particular person.
2 . The computer-implemented method of claim 1 , further comprising:
determining, by the training module of the first device, a first set of characteristics associated with the face of the first person based at least in part on the first image of the set of reference images;
determining, by the training module of the first device, a second set of characteristics associated with the face of the second person based at least in part on the second image; and
determining, by the first device, a score that corresponds to a level of similarity between the first set of characteristics associated with the face of the first person and the second set of characteristics associated with the face of the second person, wherein the first image of the first person and the second image of the second person are determined to correspond to the same person based at least in part on the score.
3 . The computer-implemented method of claim 2 , wherein determining the second set of characteristics further comprises:
determining, by the training module, a first level of confidence that the portion of the face of the second person shown in the second image is recognizable;
determining, by the training module, a second level of confidence of whether the portion of the face of the second person shown in the second image comprises the face covering; and
determining, by the training module, a faceprint of the face of the second person that corresponds to a multidimensional vector, a dimension of the multidimensional vector associated with a characteristic of the face of the second person.
4 . The computer-implemented method of claim 1 , the method further comprising:
providing, by the first device, a notification based at least in part on the determination of whether the first person is the second person, wherein the first device is a handheld device.
5 . The computer-implemented method of claim 1 , wherein the training module is trained to associate a first faceprint, generated from an image showing the non-covered face of the particular person, with a second faceprint, generated from an image that includes a simulated face covering that partially covers the face of the particular person.
6 . The computer-implemented method of claim 1 , wherein the face covering that partially covers the face of the second person corresponds to a face mask, and wherein the training module is trained based at least in part on generating simulated face masks that vary according to at least one of (I) color, (II) texture, or (III) a placement of the face mask on a face.
7 . The computer-implemented method of claim 1 , wherein the training module is trained to identify facial landmarks, the facial landmarks operable for identifying a bounding polygon, and wherein a simulated face mask is generated based at least in part on the identified bounding polygon.
8 . The computer-implemented method of claim 1 , wherein the training module is trained based at least in part on images showing simulated face coverings and images showing real face coverings, and wherein the training module is trained to perform the association between the full image of the partially covered face of the particular person and the full image of the non-covered face of the particular person based at least in part on utilizing a simulated face covering to cover the partially covered face.
9 . A first device associated with a first user, the first device comprising:
a memory configured to store computer-executable instructions; and
one or more processors in communication with the memory and configured to access the memory and execute the computer-executable instructions to, at least:
access a first image locally stored on the first device associated with the first user, the first image comprising a portion of a face of a first person, the first image being included within a set of reference images for the face of the first person based at least in part on a determination that the portion of the face of the first person shown in the first image is not covered by a face covering, and the set of reference images respectively showing non-covered portions of the face of the first person;
receive, from a second device that is a different device from the first device and the second device comprises a camera, a second image for comparison against the locally stored first image of the first person that is not covered by the face covering, the second image used by a training module of the first device to determine whether the first image of the first person and the second image of a second person correspond to a same person, the second image comprising a portion of a face of the second person, the portion comprising a face covering that partially covers the face of the second person; and
determine, using the training module of the first device, whether the first image of the first person and the second image of the second person received from the second device that is the different device from the first device corresponds to the same person, the training module being trained to associate a full image of a partially covered face of a particular person with a full image of a non-covered face of the particular person.
10 . The first device of claim 9 , wherein the instructions comprise additional instructions to:
determine, by the training module of the first device, a first set of characteristics associated with the face of the first person based at least in part on the first image of the set of reference images;
determine, by the training module of the first device, a second set of characteristics associated with the face of the second person based at least in part on the second image; and
determine, by the first device, a score that corresponds to a level of similarity between the first set of characteristics associated with the face of the first person and the second set of characteristics associated with the face of the second person, wherein the first image of the first person and the second image of the second person are determined to correspond to the same person based at least in part on the score.
11 . The first device of claim 9 , wherein the training module is trained by a remote server and the training module is subsequently deployed to a plurality of devices that includes at least one of (I) the first device or (II) a user device associated with the first device.
12 . The first device of claim 9 , wherein the first device is a resident device that receives the first image from a user device, the training module of the user device having performed the determination that the portion of the face of the first person in the first image is not covered by the face covering, and wherein the first image is transmitted by the user device to the first device based at least in part on the determination.
13 . The first device of claim 12 , wherein the first image is an image cropping that is generated from an image of a library of images stored on the user device, wherein the library of images is captured by a camera of the user device, and wherein the library of images comprises contacts associated with the user device.
14 . The first device of claim 9 , wherein the first device is a resident device, wherein the first image is received by the resident device from the second device prior to receiving the second image, the camera of the second device corresponding to an observation camera, wherein the determination that the portion of the face of the first person shown in the first image is not covered by the face covering is performed by the training module of the first device, and wherein the portion of the face of the first person is tagged as a contact associated with a user device based at least in part on the determination.
15 . The first device of claim 9 , wherein the first image is associated with a first level of image quality and the second image is associated with a second level of image quality that is different from the first level of image quality.
16 . The first device of claim 9 , wherein the first person is a contact associated with a user device, the user device being associated with the first device, and wherein the instructions comprise additional instructions to:
determine, using the training module, that the first person is not the second person; and
provide to the user device a notification that indicates that the second person is not a contact associated with the user device, wherein the notification does not include an image of the partially covered face of the second person.
17 . The first device of claim 9 , wherein the first person is a contact associated with a user device, the user device being associated with the first device, and wherein the instructions comprise additional instructions to:
receive from the second device a third image comprising a non-covered portion of the face of the second person;
determine, using the training module, that the first person is not the second person; and
provide to the user device a notification that indicates that the second person is not a contact associated with the user device, wherein the notification includes an image of the non-covered portion of the face of the second person, and wherein the notification is operable for tagging the non-covered face as another contact associated with the user device.
18 . One or more computer-readable storage media comprising computer-executable instructions that, when executed by one or more processors of a first device associated with a first user, cause the one or more processors to perform operations comprising:
accessing a first image locally stored on the first device associated with the first user, the first image comprising a portion of a face of a first person, the first image being included within a set of reference images for the face of the first person based at least in part on a determination that the portion of the face of the first person shown in the first image is not covered by a face covering, and the set of reference images respectively showing non-covered portions of the face of the first person;
receiving, from a second device that is a different device from the first device and the second device comprises a camera, a second image for comparison against the locally stored first image of the first person that is not covered by the face covering, the second image used by a training module of the first device to determine whether the first image of the first person and the second image of a second person correspond to a same person, the second image comprising a portion of a face of the second person, the portion comprising a face covering that partially covers the face of the second person; and
determining, using the training module of the first device, whether the first image of the first person and the second image of the second person received from the second device that is the different device from the first device corresponds to the same person, the training module being trained to associate a full image of a partially covered face of a particular person with a full image of a non-covered face of the particular person.
19 . The one or more computer-readable storage media of claim 18 , wherein the instructions further comprise:
determining, by the training module of the first device, a first set of characteristics associated with the face of the first person based at least in part on the first image of the set of reference images;
determining, by the training module of the first device, a second set of characteristics associated with the face of the second person based at least in part on the second image; and
determining, by the first device, a score that corresponds to a level of similarity between the first set of characteristics associated with the face of the first person and the second set of characteristics associated with the face of the second person, wherein the first image of the first person and the second image of the second person are determined to correspond to the same person based at least in part on the score.
20 . The one or more computer-readable storage media of claim 19 , wherein the instructions further comprise:
maintaining a face quality metric that indicates a level of quality associated with the second set of characteristics associated with the face of the second person, the face quality metric being operable for determining whether a particular face of a person is recognizable or unrecognizable; and
determining that the face of the second person is recognizable based at least in part determining that the level of quality indicated by the face quality metric matches a threshold; and determining whether the first person is the second person based at least in part on determining that the face of the second person is recognizable.
21 . The one or more computer-readable storage media of claim 18 , wherein the training module is trained to associate a first faceprint, generated from an image showing a real or simulated face covering that partially covers a face of a person, with a second faceprint, generated from another image that shows a non-covered face of the person.
22 . The one or more computer-readable storage media of claim 18 , wherein the first image is received from a remote server, the first image having been first transmitted from a user device associated with the first device to the remote server, wherein the first image is encrypted, and wherein an encryption key operable for decrypting the first image is shared with the first device and not shared with the remote server.