Apparatus for leveling person image and operating method thereof

1. A method of performing leveling of a person image, the method comprising:

receiving an original person image;

selecting an arbitrary latent vector in a latent space;

generating a virtual person image based on the latent vector;

activating an original person region and deactivating an original background region from among the original person region and the original background region included in the original person image;

activating a virtual person region and deactivating a virtual background region from among the virtual person region and the virtual background region included in the virtual person image;

optimizing the latent vector such that identity similarity between an image of the original person region and an image of the virtual person region increases;

manipulating the optimized latent vector; and

generating a levelled person image, in which a person stares forward and shows a neutral emotion, and in which an illumination is illuminating a front of the person's face, corresponding to the original person image, by using the manipulated latent vector.

2. The method of claim 1 , wherein the optimizing comprises calculating a loss between the original person image and the virtual person image and optimizing the latent vector in a direction in which the loss is reduced.

3. The method of claim 1 , wherein the optimizing comprises:

generating a binary mask by extracting a boundary line between the original person region and the original background region included in the original person image;

by applying the binary mask to the original person image and the virtual person image, activating the original person region and deactivating the original background region from the original person image, and activating the virtual person region and deactivating the virtual background region from the virtual person image; and

optimizing the latent vector such that identity similarity between an image of the original person region and an image of the virtual person region increases.

4. The method of claim 3 , wherein the generating of the binary mask comprises generating a softened binary mask by softening the boundary line of the binary mask.

5. The method of claim 1 , wherein the manipulating comprises:

obtaining subspaces respectively corresponding to a plurality of semantic attributes of the original person image in the latent space; and

manipulating the optimized latent vector by linearly transforming the subspaces to move values of the subspaces by a certain value.

6. The method of claim 5 , wherein the values of the subspaces are moved by the certain value by using a grid search method.

7. The method of claim 5 , wherein the semantic attributes comprise at least one of a pose attribute, an expression attribute, and an illumination attribute.

8. The method of claim 1 , further comprising excluding the original person image when a face in the original person image is blocked by a certain ratio or more, or when the eyes and the mouth in the original person image are not aligned.

9. A computer-readable recording medium having recorded thereon a program for executing the method of claim 1 on a computer.

10. An apparatus for performing leveling of a person image, the apparatus comprising:

a memory storing at least one program; and

a processor configured to perform an operation by executing the at least one program,

wherein the processor is configured to:

receive an original person image including a person object,

select an arbitrary latent vector in a latent space,

generate a virtual person image based on the latent vector,

activate an original person region and deactivating an original background region from among the original person region and the original background region included in the original person image,

activate a virtual person region and deactivating a virtual background region from among the virtual person region and the virtual background region included in the virtual person image,

optimize the latent vector such that identity similarity between an image of the original person region and an image of the virtual person region increases,

manipulate the optimized latent vector, and

generate a levelled person image, in which a person stares forward and shows a neutral emotion, and in which an illumination is illuminating a front of the person's face, corresponding to the original person image, by using the manipulated latent vector.

11. A method of generating a character face image, the method comprising:

receiving an original person image from a user terminal;

selecting an arbitrary latent vector in a latent space;

generating a virtual person image based on the latent vector;

activating an original person region and deactivating an original background region from among the original person region and the original background region included in the original person image;

activating a virtual person region and deactivating a virtual background region from among the virtual person region and the virtual background region included in the virtual person image;

optimizing the latent vector such that identity similarity between an image of the original person region and an image of the virtual person region increases;

manipulating the optimized latent vector;

converting the original person image into a levelled person image, in which a person stares forward and shows a neutral emotion, and in which an illumination is illuminating a front of the person's face, by using the manipulated latent vector; and

generating the character face image to correspond to the levelled person image.

12. The method of claim 11 , wherein the converting comprises:

determining whether the original person image is in a levelled state; and

converting the original person image into the levelled person image based on the determination.

13. The method of claim 11 , further comprising excluding the original person image when a face in the original person image is blocked by a certain ratio or more, or when the eyes and the mouth in the original person image are not aligned.

14. The method of claim 11 , wherein the optimizing comprises:

generating a binary mask by extracting a boundary line between the original person region and the original background region included in the original person image; and

by applying the binary mask to the original person image and the virtual person image, activating the original person region and deactivating the original background region from the original person image, and activating the virtual person region and deactivating the virtual background region from the virtual person image.

15. A game server for generating a character face image, the game server comprising:

a communication unit;

a memory storing at least one program; and

a processor configured to perform an operation by executing the at least one program,

wherein the communication unit receives an original person image from a user terminal, and

wherein the processor is configured to:

select an arbitrary latent vector in a latent space,

generate a virtual person image based on the latent vector,

activate an original person region and deactivating an original background region from among the original person region and the original background region included in the original person image,

activate a virtual person region and deactivating a virtual background region from among the virtual person region and the virtual background region included in the virtual person image,

optimize the latent vector such that identity similarity between an image of the original person region and an image of the virtual person region increases,

manipulate the optimized latent vector,

convert the original person image into a levelled person image, in which a person stares forward and shows a neutral emotion, and in which an illumination is illuminating a front of the person's face, by using the manipulated latent vector, and

generate the character face image to correspond to the levelled person image.