Low-resolution face recognition device and low-resolution face recognizer learning device and method

1 . A face image recognizer learning device comprising:

an image inputter configured to input a high-resolution learning image and a low-resolution learning image each including a pre-registered face;

a low-resolution image generator configured to generate a low-resolution converted image from the high-resolution learning image input by the image inputter;

a low-resolution face feature extractor configured to extract a high-resolution face feature from the high-resolution learning image input by the image inputter and to extract a low-resolution face feature from the generated low-resolution converted image;

a low-resolution image classifier configured to distinguish the low-resolution converted image generated through the low-resolution image generator and the low-resolution learning image input by the image inputter from each other;

a high-resolution image generator configured to generate a high-resolution converted image by using the low-resolution converted image generated through the low-resolution image generator;

a high-resolution face feature extractor configured to extract a first high-resolution face feature from the high-resolution converted image generated through the high-resolution image generator and to extract a second high-resolution face feature from the high-resolution learning image input by the image inputter; and

a high-resolution image classifier configured to distinguish the high-resolution converted image generated through the high-resolution image generator and the high-resolution learning image input by the image inputter from each other;

a first perceptual loss function for distinguishing a same person by making the extracted low-resolution face feature and at least one of the first high-resolution face feature or the second high-resolution face feature equal to each other;

a first GAN loss function for processing the generated low-resolution converted image and low-resolution learning image input by the image inputter so that the generated low-resolution converted image and the low-resolution learning image input by the image inputter are not distinguished from each other; and

a first pixel consistency loss function for processing pixel values of the high-resolution learning image input by the image inputter and the generated low-resolution converted image so that the pixel values are consistent with each other;

wherein the low-resolution image generator is learned through the first perceptual loss function, the first GAN loss function, and the first pixel consistency loss function;

a second perceptual loss function to distinguish the same person by making the first high-resolution face feature and the second high-resolution face feature equal to each other;

a second GAN loss function for separating the generated high-resolution converted image and the high-resolution learning image input by the image inputter from each other; and

a second pixel consistency loss function for processing pixel values of the generated high-resolution converted image and the high-resolution learning image input by the image inputter so that the pixel values are consistent with each other;

wherein the high-resolution image generator is learned through the second perceptual loss function, the second GAN loss function and the second pixel consistency loss function; and

wherein the low-resolution image classifier is learned by using the first GAN loss function.

2 . A face image recognizer learning method comprising:

inputting, by an image inputter, a high-resolution learning image and a low-resolution learning image each including a pre-registered face;

generating, by a low-resolution image generator, a low-resolution converted image from the high-resolution learning image input by the image inputter;

extracting, by a low-resolution face feature extractor, a high-resolution face feature from the high-resolution learning image input by the image inputter, and extracting a low-resolution face feature from the generated low-resolution converted image;

distinguishing, by a low-resolution image classifier, the generated low-resolution converted image and the low-resolution learning image input by the image inputter from each other;

generating, by a high-resolution image generator, a high-resolution converted image by using the generated low-resolution converted image;

extracting, by a high-resolution face feature extractor, a first high-resolution face feature from the generated high-resolution converted image, and extracting a second high-resolution face feature from the high-resolution learning image input by the image inputter;

distinguishing, by a high-resolution image classifier, the generated high-resolution converted image and the high-resolution learning image input by the image inputter from each other;

using a first perceptual loss function to distinguish a same person by making the extracted low-resolution face feature and at least one of the first high-resolution face feature or the second high-resolution face feature equal to each other;

processing, by a first GAN loss function, the generated low-resolution converted image and low-resolution learning image input by the image inputter so that the generated low-resolution converted image and the low-resolution learning image input by the image inputter are not distinguished from each other;

processing, by a first pixel consistency loss function, pixel values of the high-resolution learning image input by the image inputter and the generated low-resolution converted image so that the pixel values are consistent with each other;

learning the low-resolution image generator through the first perceptual loss function, the first GAN loss function, and the first pixel consistency loss function;

using a second perceptual loss function to distinguish the same person by making the first high-resolution face feature and the second high-resolution face feature equal to each other;

processing, by a second GAN loss function, the generated high-resolution converted image and the high-resolution learning image input by the image inputter so that the generated high-resolution converted image and the high-resolution learning image input by the image inputter are not distinguished from each other;

processing, by a second pixel consistency loss function, pixel values of the generated high-resolution converted image and the high-resolution learning image input by the image inputter so that the pixel values consistent with each other; and

learning the high-resolution image generator through the second perceptual loss function, the second GAN loss function, and the second pixel consistency loss function.

3 . The face image recognizer learning method of claim 2 , comprising learning the low-resolution image classifier by using the first GAN loss function.

4 . The face image recognizer learning method of claim 2 , comprising learning the high-resolution image classifier by the second GAN loss function.

5 . The face image recognizer learning method of claim 2 , comprising:

inputting, by a high-resolution learning image inputter, a pre-registered high-resolution learning face image;

generating, by a low-resolution image face generator, a low-resolution converted face image by using the high-resolution learning face image input by the high-resolution learning image inputter;

extracting, by a low-resolution face feature extractor, a low-resolution face feature from the generated low-resolution converted face image;

extracting, by a high-resolution face feature extractor, a third high-resolution face feature from the high-resolution learning face image input by the high-resolution learning image inputter;

extracting, by a face quality feature extractor, face quality features by using the input high-resolution face image input by the high-resolution learning image inputter and the generated low-resolution converted face image, computing respective extracted face quality features with a predetermined function, and providing a face quality resultant value depending on whether a computation result is a high-resolution image or a low-resolution image;

merging, by a channel embedder, a high-resolution concatenate value, obtained by concatenating the third high-resolution face feature and the face quality resultant value calculated through the face quality feature extractor, with a low-resolution concatenate value obtained by concatenating the low-resolution face feature and the face quality resultant value calculated through the face quality feature extractor; and

generating, by a feature adaptation network, a feature of an adapted resolution and a quality face feature map by using the high-resolution concatenate value and the low-resolution concatenate value.

6 . The face image recognizer learning method of claim 5 , wherein the generating of the low-resolution converted face image generates a low-resolution converted image paired with a high-resolution face image, and extracts face features from the low-resolution converted image and the high-resolution face image.

7 . The face image recognizer learning method of claim 5 , wherein the extracting of the face quality features calculates a face quality resultant value in accordance with a corresponding resolution by performing a SoftMax function operation (Qs) on an extracted face quality feature.