Method and apparatus for generating 3D virtual viewpoint image

1. A method for generating a three-dimension (3D) virtual viewpoint image, the method comprising:

segmenting a first image into a plurality of images indicating different layers based on depth information of the first image at a gaze point of a user;

inpainting an area occluded by foreground in the plurality of images based on depth information of a reference viewpoint image;

providing a parallax to a second image in which the plurality of images are sequentially arranged based on depth of a gaze direction of the user by determining a third image generated when a central axis of the user in the second image is moved in at least one direction of up, down, left, right, forward, and backward; and

performing image warping on the third image by separating a background area and a foreground area from the third image, and performing the image warping on the background area and the foreground area.

2. The method of claim 1 , wherein:

the performing image warping on the background area and the foreground area comprises

performing the image warping in an order from the farthest foreground area to the closest foreground area among the foreground areas.

3. An apparatus for generating a three-dimension (3D) virtual viewpoint image, the apparatus comprising:

an image information generator configured to segment a first image into a plurality of images indicating different layers based on depth information of the first image at a gaze point of a user;

an image generator configured to inpaint an area occluded by foreground in the plurality of images based on depth information of a reference viewpoint image; and

a parallax generator configured to

provide a parallax to a second image in which the plurality of images are sequentially arranged based on depth of a gaze direction of the user,

determine a third image generated when a central axis of the user in the second image is moved in at least one direction of up, down, left, right, forward, and backward,

perform image warping on the third image,

separate a background area and a foreground area from the third image, and

perform image warping on the background area and the foreground area.

4. The apparatus of claim 3 , wherein when the parallax generator performs the image warping on the background area and the foreground area, the parallax generator further configured to

perform the image warping in an order from the farthest foreground area from the user to the closest foreground area among the foreground areas.

5. An apparatus for generating a three-dimensional (3D) virtual viewpoint image, the apparatus comprising:

a processor and a memory, wherein the processor executes a program stored in the memory to perform:

segmenting a first image into a plurality of images representing different layers based on depth information of the first image at a gaze point of a user;

reconstructing an area occluded by foreground in the plurality of images based on depth information of a reference viewpoint image;

providing a parallax to a second image in which the plurality of images are sequentially arranged based on depth of a gaze direction of the user;

determining a third image generated when a central axis of the user in the second image is moved in at least one direction of up, down, left, right, forward, and backward; and

performing image warping on the third image by separating a background area and a foreground area from the third image, and performing the image warping on the background area and the foreground area.

6. The apparatus of claim 5 ,

wherein when the processor performs performing the image warping on the background area and the foreground area, the processor performs

performing the image warping in an order from the farthest foreground area from the user to the closest foreground area among the foreground areas.