Method and system for increasing face images
The present invention provides a method for increasing face images. The method includes: obtaining a face image, and determining whether the face image belongs to a frontal direction or a side direction; transforming the face image into a feature vector; performing a facial attribute editing algorithm according to the feature vector to generate a first generation vector which is different from the face image in at least one attribute; if the face image belongs to the side direction, performing a facial pose conversion algorithm according to the feature vector to generate a second generation vector which is different from the face image in pose directions; inputting the first generation vector or the second generation vector to a generative adversarial network to output at least one generated face image; and adding the face image and the generated face image into a database.
1. A method performed by a computer system, the method comprising:
capturing a face image and determining whether the face image belongs to a front direction or a side direction;
transforming the face image into a feature vector;
performing a facial attribute editing algorithm according to the feature vector to generate a first generation vector, wherein at least one attribute of the first generation vector is different from that of the face image;
if the face image belongs to the side direction, performing a facial pose conversion algorithm according to the feature vector to generate a second generation vector, wherein a pose direction of the second generation vector is different from that of the face image;
inputting the first generation vector or the second generation vector into a generative adversarial network to output at least one generated face image; and
adding the face image and the at least one generated face image into a database,
wherein determining whether the face image belongs to the front direction or the side direction comprising:
flipping the face image along a central axis to obtain a flipped image;
calculating a structural similarity index between the face image and the flipped image; and
if the structural similarity index is less than a threshold, determining that the face image belongs to the side direction, otherwise determining that the face image belongs to the front direction.
2. The method of claim 1 , wherein the step of transforming the face image into the feature vector comprises:
inputting the face image into an encoder to obtain an initial feature vector;
inputting the initial feature vector and a noise vector into a generator to obtain a reconstructed image;
calculating a loss function between the face image and the reconstructed image; and
adjusting the initial feature vector and the noise vector according to the loss function for a plurality of iterations, and setting the initial feature vector corresponding to a minimum of the loss function among the iterations as the feature vector.
3. The method of claim 1 , further comprising:
determining if the at least one generated face image belongs to human face according to a first classifier, and if not, deleting the at least one generated face image; and
determining if the face image and the at least one generated face image belongs to a same person according to a recognizer, and if not, deleting the at least one generated face image.
4. The method of claim 1 , further comprising:
determining if the at least one generated face image has a predetermined attribute according to a second classifier, and if not, deleting the at least one generated face image.
5. A system comprising:
an image capture device configured to obtain a face image; and
a computer system communicatively connected to the image capture device and configured to perform a plurality of steps:
obtaining the face image and determining whether the face image belongs to a front direction or a side direction;
transforming the face image into a feature vector;
performing a facial attribute editing algorithm according to the feature vector to generate a first generation vector, wherein at least one attribute of the first generation vector is different from that of the face image;
if the face image belongs to the side direction, performing a facial pose conversion algorithm according to the feature vector to generate a second generation vector, wherein a pose direction of the second generation vector is different from that of the face image;
inputting the first generation vector or the second generation vector into a generative adversarial network to output at least one generated face image; and
adding the face image and the at least one generated face image into a database,
wherein determining whether the face image belongs to the front direction or the side direction comprising:
flipping the face image along a central axis to obtain a flipped image;
calculating a structural similarity index between the face image and the flipped image; and
if the structural similarity index is less than a threshold, determining that the face image belongs to the side direction, otherwise determining that the face image belongs to the front direction.
6. The system of claim 5 , wherein the step of transforming the face image into the feature vector comprises:
inputting the face image into an encoder to obtain an initial feature vector;
inputting the initial feature vector and a noise vector into a generator to obtain a reconstructed image;
calculating a loss function between the face image and the reconstructed image; and
adjusting the initial feature vector and the noise vector according to the loss function for a plurality of iterations, and setting the initial feature vector corresponding to a minimum of the loss function among the iterations as the feature vector.
7. The system of claim 5 , wherein the plurality of steps further comprise:
determining if the at least one generated face image belongs to human face according to a first classifier, and if not, deleting the at least one generated face image; and
determining if the face image and the at least one generated face image belongs to a same person according to a recognizer, and if not, deleting the at least one generated face image.
8. The system of claim 5 , wherein the plurality of steps further comprise:
determining if the at least one generated face image has a predetermined attribute according to a second classifier, and if not, deleting the at least one generated face image.
9. A non-transitory computer readable storage medium storing a plurality of instructions which are configured to perform a plurality of steps:
capturing a face image and determining whether the face image belongs to a front direction or a side direction;
transforming the face image into a feature vector;
performing a facial attribute editing algorithm according to the feature vector to generate a first generation vector, wherein at least one attribute of the first generation vector is different from that of the face image;
if the face image belongs to the side direction, performing a facial pose conversion algorithm according to the feature vector to generate a second generation vector, wherein a pose direction of the second generation vector is different from that of the face image;
inputting the first generation vector or the second generation vector into a generative adversarial network to output at least one generated face image; and
adding the face image and the at least one generated face image into a database,
wherein determining whether the face image belongs to the front direction or the side direction comprising:
flipping the face image along a central axis to obtain a flipped image;
calculating a structural similarity index between the face image and the flipped image; and
if the structural similarity index is less than a threshold, determining that the face image belongs to the side direction, otherwise determining that the face image belongs to the front direction.
10. The non-transitory computer readable storage medium of claim 9 , wherein the step of transforming the face image into the feature vector comprises:
inputting the face image into an encoder to obtain an initial feature vector;
inputting the initial feature vector and a noise vector into a generator to obtain a reconstructed image;
calculating a loss function between the face image and the reconstructed image; and
adjusting the initial feature vector and the noise vector according to the loss function for a plurality of iterations, and setting the initial feature vector corresponding to a minimum of the loss function among the iterations as the feature vector.
11. The non-transitory computer readable storage medium of claim 9 , wherein the plurality of steps further comprise:
determining if the at least one generated face image belongs to human face according to a first classifier, and if not, deleting the at least one generated face image; and
determining if the face image and the at least one generated face image belongs to a same person according to a recognizer, and if not, deleting the at least one generated face image.
12. The non-transitory computer readable storage medium of claim 9 , wherein the plurality of steps further comprise:
determining if the at least one generated face image has a predetermined attribute according to a second classifier, and if not, deleting the at least one generated face image.