Method and apparatus with face recognition
A method with face recognition includes: determining a first global feature of a first face image and a first global feature of a second face image based on a local feature of the first face image and a local feature of the second face image, respectively; determining a final global feature of the first face image based on the first global feature of the first face image and a second global feature of the first face image; determining a final global feature of the second face image based on the first global feature and a second global feature of the second face image; and recognizing the first face image and the second face image based on the final global feature of the first face image and the final global feature of the second face image, wherein the second global feature of the first face image is determined based on the local feature of the first face image, and the second global feature of the second face image is determined based on the local feature of the second face image.
1 . A method with face recognition, the method comprising:
determining a first global feature of a first face image and a first global feature of a second face image based on a local feature of the first face image and a local feature of the second face image, respectively;
determining a final global feature of the first face image based on the first global feature of the first face image and a second global feature of the first face image;
determining a final global feature of the second face image based on the first global feature and a second global feature of the second face image; and
recognizing the first face image and the second face image based on the final global feature of the first face image and the final global feature of the second face image,
wherein the second global feature of the first face image is determined based on the local feature of the first face image, and the second global feature of the second face image is determined based on the local feature of the second face image,
wherein the determining of the first global feature of a first face image and the first global feature of a second face image comprises:
determining a first attention pooling feature of the first face image with respect to the second face image and a second attention pooling feature of the second face image with respect to the first face image, based on the local feature of the first face image and the local feature of the second face image, respectively, using a multi-head attention mechanism;
determining the first global feature of the first face image based on the first attention pooling feature; and
determining the first global feature of the second face image based on the second attention pooling feature.
2 . The method of claim 1 , wherein the determining of the first global feature of the first face image and the first global feature of the second face image based on the local feature of the first face image and the local feature of the second face image, respectively, comprises performing an attention pooling processing based on the local feature of the first face image and the local feature of the second face image to determine the first global feature of the first face image and the first feature of the second face image, respectively.
3 . The method of claim 2 , wherein
the local feature of the first face image is a local feature of the first face image in which position information is embedded, and
the local feature of the second face image is a local feature of the second face image in which position information is embedded.
4 . The method of claim 1 , wherein the determining of the first attention pooling feature of the first face image with respect to the second face image and the second attention pooling feature of the second face image with respect to the first face image using the multi-head attention mechanism, based on the local feature of the first face image and the local feature of the second face image comprises:
determining a first query vector by multiplying the local feature of the second face image by a first query matrix corresponding to a query, determining a first key vector by multiplying the local feature of the first face image by a first key matrix corresponding to a key, determining a first value vector by multiplying the local feature of the first face image by a first value matrix corresponding to a value, and determining an attention pooling feature of the first face image with respect to the second face image based on the determined first query vector, the determined first key vector and the determined first value vector; and
determining a second query vector by multiplying the local feature of the first face image by a second query matrix corresponding to a query, determining a second key vector by multiplying the local feature of the second face image by a second key matrix corresponding to a key, determining a second value vector by multiplying the local feature of the second face image by a second value matrix corresponding to a value, and determining the second attention pooling feature of the second face image with respect to the first face image based on the determined second query vector, the determined second key vector and the determined second value vector.
5 . The method of claim 1 , wherein the determining of the first global feature of the first face image and the first global feature of the second face image comprises determining the first global feature of the first face image and the first global feature of the second face image, respectively, using a same network or different networks.
6 . A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 1 .
7 . An apparatus with face recognition, the apparatus comprising:
one or more processors comprising:
a first global feature obtaining module configured to determine a first global feature of a first face image and a first global feature of a second face image based on a local feature of the first face image and a local feature of the second face image, respectively;
a final global feature obtaining module configured to determine a final global feature of the first face image based on the first global feature and a second global feature of the first face image and determine a final global feature of the second face image based on the first global feature and a second global feature of the second face image; and
a recognition module configured to recognize the first face image and the second face image based on the final global feature of the first face image and the final global feature of the second face image,
wherein the second global feature of the first face image is determined based on the local feature of the first face image, and the second global feature of the second face image is determined based on the local feature of the second face image,
wherein, for the determining of the first global feature of the first face image and the first global feature of the second face image, the first global feature obtaining module is configured to:
determine a first attention pooling feature of the first face image with respect to the second face image and a second attention pooling feature of the second face image with respect to the first face image, based on the local feature of the first face image and the local feature of the second face image, respectively, using a multi-head attention mechanism;
determine the first global feature of the first face image based on the first attention pooling feature; and
determine the first global feature of the second face image based on the second attention pooling feature.
8 . The apparatus of claim 7 , wherein, for the determining of the first global feature of the first face image and the first global feature of the second face image, the first global feature obtaining module is configured to perform an attention pooling processing based on the local feature of the first face image and the local feature of the second face image to determine the first global feature of the first face image and the first global feature of the second face image, respectively.
9 . The apparatus of claim 8 , wherein
the local feature of the first face image is a local feature of the first face image in which position information is embedded, and
the local feature of the second face image is a local feature of the second face image in which position information is embedded.
10 . The apparatus of claim 7 , wherein
the first global feature obtaining module comprises a cross-attention module, and
for the determining of the first attention pooling feature of the first face image and the second attention pooling feature of the second face image, the cross-attention module is configured to:
determine a first query vector by multiplying the local feature of the second face image by a first query matrix corresponding to a query, determine a first key vector by multiplying the local feature of the first face image by a first key matrix corresponding to a key, determine a first value vector by multiplying the local feature of the first face image by a first value matrix corresponding to a value, and determine the first attention pooling feature of the first face image with respect to the second face image based on the determined first query vector, the determined first key vector and the determined first value vector; and
determine a second query vector by multiplying the local feature of the first face image by a second query matrix corresponding to a query, determine a second key vector by multiplying the local feature of the second face image by a second key matrix corresponding to a key, determine a second value vector by multiplying the local feature of the second face image by a second value matrix corresponding to a value, and determine the second attention pooling feature of the second face image with respect to the first face image based on the determined second query vector, the determined second key vector and the determined second value vector.
11 . The apparatus of claim 7 , wherein the first global feature obtaining module comprises:
a first face image first global feature determining module configured to determine the first global feature of the first face image; and
a second face image first global feature determining module configured to determine the first global feature of the second face image.
12 . An electronic device comprising:
one or more processors configured to:
determine a first global feature of a first face image and a first global feature of a second face image based on a local feature of the first face image and a local feature of the second face image, respectively,
determine a final global feature of the first face image based on the first global feature and a second global feature of the first face image,
determine a final global feature of the second face image based on the first global feature and a second global feature of the second face image, and
recognize the first face image and the second face image based on the final global feature of the first face image and the final global feature of the second face image,
wherein the second global feature of the first face image is determined based on the local feature of the first face image, and the second global feature of the second face image is determined based on the local feature of the second face image,
wherein the one or more processors are configured to:
determine a first attention pooling feature of the first face image with respect to the second face image and a second attention pooling feature of the second face image with respect to the first face image, based on the local feature of the first face image and the local feature of the second face image, respectively, using a multi-head attention mechanism;
determine the first global feature of the first face image based on the first attention pooling feature; and
determine the second global feature of the second face image based on the second attention pooling feature.
13 . The electronic device of claim 12 , wherein, for the determining of the first global feature of the first face image and the first global feature of the second face image, the one or more processors are configured to perform an attention pooling processing based on the local feature of the first face image and the local feature of the second face image to determine the first global feature of the first face image and the first global feature of the second face image, respectively.
14 . The electronic device of claim 13 , wherein
the local feature of the first face image is a local feature of the first face image in which position information is embedded, and
the local feature of the second face image is a local feature of the second face image in which position information is embedded.
15 . The electronic device of claim 12 , wherein, for the determining of the first attention pooling feature of the first face image and the second attention pooling feature of the second face image, the one or more processors are configured to:
determine a first query vector by multiplying the local feature of the second face image by a first query matrix corresponding to a query, determine a first key vector by multiplying the local feature of the first face image by a first key matrix corresponding to a key, determine a first value vector by multiplying the local feature of the first face image by a first value matrix corresponding to a value, and determine the first attention pooling feature of the first face image with respect to the second face image based on the determined first query vector, the determined first key vector and the determined first value vector; and
determine a second query vector by multiplying the local feature of the first face image by a second query matrix corresponding to a query, determine a second key vector by multiplying the local feature of the second face image by a second key matrix corresponding to a key, determine a second value vector by multiplying the local feature of the second face image by a second value matrix corresponding to a value, and determine the second attention pooling feature of the second face image with respect to the first face image based on the determined second query vector, the determined second key vector and the determined second value vector.
16 . The apparatus of claim 12 , further comprising a memory storing instructions that, when executed by the one or more processors, configure the one or more processors to perform the determining of the first global feature of the first face image and the first global feature of the second face image, the determining of the final global feature of the first face image, the determining of the final global feature of the second face image, and the recognizing of the first face image and the second face image.