Method and apparatus for rendering light field image
Provided is an electronic device, wherein the at least one processor is configured to execute the at least one instruction to obtain a first light field image including view images with a first number of views, obtain, from the first light field image, a second light field image including view images with a second number of views, obtain first location information corresponding to each of sub-pixels in the second light field image, obtain a first layer image by inputting the second light field image and the first location information to an artificial intelligence model configured to perform factorization, obtain a third light field image including view images with a third number of views by inputting the first layer image to a simulation model, and train the artificial intelligence model, based on a result of comparing the first light field image and the third light field image with each other.
1 . An electronic device comprising:
a display comprising a plurality of stacked panels;
a memory storing at least one instruction; and
at least one processor,
wherein the at least one processor is configured to execute the at least one instruction to:
obtain a first light field image including view images with a first number of views;
perform masking or interpolation on the first light field with the first number of views to obtain a second light field image including view images with a second number of views;
obtain first location information corresponding to locations within the second light field image of sub-pixels included in each view image of the second light field image;
obtain a first layer image by inputting the second light field image and the first location information to an artificial intelligence model configured to perform factorization, the first layer image comprising a plurality of layer images corresponding to the plurality of stacked panels;
obtain a third light field image including view images with a third number of views, by inputting the first layer image to a simulation model configured to restore a light field image from a layer image; and
train the artificial intelligence model, based on a result of comparing the first light field image and the third light field image.
2 . The electronic device of claim 1 ,
wherein the first number of views is different from the second number of views.
3 . The electronic device of claim 2 , wherein, during the obtaining of the second light field image with the second number of views by performing masking on the first light field image, the at least one processor is further configured to:
determine the second number of views, based on a disparity between views of the first light field image;
generate a mask, based on the second number of views; and
perform masking on the first light field image, based on the generated mask.
4 . The electronic device of claim 2 , wherein, during the obtaining of the second light field image with the second number of views by performing masking on the first light field image, the at least one processor is further configured to:
determine the second number of views, based on a user input;
generate a mask, based on the second number of views; and
perform masking on the first light field image, based on the generated mask.
5 . The electronic device of claim 2 , wherein, during the obtaining of the second light field image with the second number of views by performing interpolation on the first light field image, the at least one processor is further configured to perform interpolation on a sub-sampled light field image obtained by performing sub-sampling on the first light field image.
6 . The electronic device of claim 1 , wherein the at least one processor is further configured to:
obtain a fourth light field image with a fourth number of views;
when the fourth number of views is less than a number of views corresponding to a viewing angle of a display, obtain a fifth light field image with a same number of formats as the number of views corresponding to the viewing angle of the display by changing a format of the fourth light field image;
obtain second location information corresponding to each sub-pixel of a plurality of sub-pixels in the fourth light field image or the fifth light field image; and
obtain a second layer image by inputting one of the fourth light field image and the fifth light field image and the second location information to the trained artificial intelligence model.
7 . The electronic device of claim 6 , wherein, during the obtaining of the fifth light field image, the at least one processor is further configured to perform masking on the fourth light field image to obtain the fifth light field image when a disparity between views of the fourth light field image is not uniform.
8 . The electronic device of claim 6 , wherein, during the obtaining of the fifth light field image, the at least one processor is further configured to perform masking or interpolation on the fourth light field image to obtain the fifth light field image when a disparity between views of the fourth light field image is uniform.
9 . The electronic device of claim 6 , further comprising the display, and
wherein the at least one processor is further configured to render the second layer image on the display.
10 . The electronic device of claim 6 , further comprising a transceiver, and
wherein the at least one processor is further configured to transmit the second layer image to the display through the transceiver, and
the second layer image is rendered on the display.
11 . A method performed by an electronic device that includes a display including a plurality of stacked panels, the method comprising:
obtaining a first light field image including view images with a first number of views;
obtaining, from the first light field image, a second light field image including view images with a second number of views, wherein the obtaining the second light field image comprises masking the first light field image with the first number of views or interpolating the first light field image with the first number of views;
obtaining first location information corresponding to locations within the second light field image of sub-pixels included in each view image of the second light field image;
obtaining a first layer image by inputting the second light field image and the first location information to an artificial intelligence model configured to perform factorization, the first layer image including a plurality of layer images corresponding to the plurality of stacked panels;
obtaining a third light field image including view images with a third number of views, by inputting the first layer image to a simulation model configured to restore a light field image from a layer image; and
training the artificial intelligence model, based on a result of comparing the first light field image and the third light field image.
12 . The method of claim 11 , wherein
the first number of views is different from the second number of views.
13 . The method of claim 12 , wherein the obtaining of the second light field image with the second number of views by performing masking on the first light field image comprises:
determining the second number of views, based on a disparity between the views of the first light field image;
producing a mask, based on the second number of views; and
performing masking on the first light field image, based on the generated mask.
14 . The method of claim 12 , wherein the obtaining of the second light field image with the second number of views by performing masking on the first light field image comprises:
determining the second number of views, based on a user input;
producing a mask, based on the second number of views; and
performing masking on the first light field image, based on the generated mask.
15 . The method of claim 12 , wherein the obtaining of the second light field image with the second number of views by interpolating the first light field image comprises interpolating a sub-sampled light field image obtained by performing sub-sampling on the first light field image.
16 . The method of claim 11 , further comprising:
obtaining a fourth light field image with a fourth number of views;
when the fourth number of views is less than a number of views corresponding to a viewing angle of a display, obtaining a fifth light field image with a same number of formats as the number of views corresponding to the viewing angle of the display by changing a format of the fourth light field image;
obtaining second location information corresponding to each sub-pixel of a plurality of sub-pixels in the fourth light field image or the fifth light field image; and
obtaining a second layer image by inputting one of the fourth light field image and the fifth light field image and the second location information to the trained artificial intelligence model.
17 . The method of claim 16 , wherein the obtaining of the fifth light field image comprises obtaining the fifth light field image by performing masking on the fourth light field image when a disparity between the views of the fourth light field image is not uniform.
18 . The method of claim 16 , wherein the obtaining of the fifth light field image comprises obtaining the fifth light field image by performing masking or interpolation on the fourth light field image when a disparity between the views of the fourth light field image is uniform.
19 . The method of claim 16 , further comprising rendering the second layer image on the display.
20 . A non-transitory computer-readable recording medium of an electronic device that includes a display including a plurality of stacked panels, the non-transitory computer-readable recording medium having recorded thereon a program for causing a computer to perform a method comprising:
obtaining a first light field image including view images with a first number of views;
obtaining, from the first light field image, a second light field image including view images with a second number of views, wherein the obtaining the second light field image comprises masking the first light field image with the first number of views or interpolating the first light field image with the first number of views;
obtaining first location information corresponding to locations within the second light field image of sub-pixels included in each view image of the second light field image;
obtaining a first layer image by inputting the second light field image and the first location information to an artificial intelligence model configured to perform factorization, the first layer image including a plurality of layer images corresponding to the plurality of stacked panels;
obtaining a third light field image including view images with a third number of views, by inputting the first layer image to a simulation model configured to restore a light field image from a layer image; and
training the artificial intelligence model, based on a result of comparing the first light field image and the third light field image.