Display apparatus and the control method thereof

1. A display apparatus comprising:

a display comprising:

a first display panel comprising a plurality of sub-pixels,

a lens array disposed on the first display panel and comprises a plurality of unit lens, and

a second display panel disposed on the lens array; and

a processor configured to:

based on a plurality of light field (LF) images, obtain a left (L) image and a right (R) image to drive the display by time-multiplexing,

correct a second L image to drive the second display panel among L images based on a first R image to drive the first display panel among R images, and correct a second R image to drive the second display panel among the R images based on a first L image to drive the first display panel among the L images, and

display a stereoscopic image by driving the display by time-multiplexing based on the L image which comprises the corrected second L image and the R image which comprises the corrected second R image,

wherein the lens array disposed on the first display panel such that respectively unit lens is arranged on a predetermined number of first sub-pixels, and

wherein the processor is further configured to,

based on information of a viewpoint at which a plurality of LF images are captured, extract an LF image representing a left viewpoint and an LF image representing a right viewpoint among the plurality of LF images,

based on the LF image representing the left viewpoint obtain a left (L) images respectively corresponding to the first display panel and the second display panel, and based on the LF image representing the right viewpoint obtain a right (R) images respectively corresponding to the first display panel and the second display panel.

2. The display apparatus of claim 1 , wherein the processor is further configured to:

obtain an R crosstalk image based on the first R image and an L crosstalk image based on the first L image;

correct the second L image based on a pixel value of the R crosstalk image; and

correct the second R image based on a pixel value of the L crosstalk image.

3. The display apparatus of claim 2 , wherein the processor is further configured to:

correct the second L image to have a pixel value reduced as much as a pixel value of the R crosstalk image with respect to a pixel of a same position; and

correct the second R image to have a pixel value reduced as much as a pixel value of the L crosstalk image with respect to a pixel of a same position.

4. The display apparatus of claim 2 , wherein the processor is further configured to:

obtain the R crosstalk image having a pixel value obtained by multiplying a crosstalk ratio by the pixel value of the first R image; and

obtain the L crosstalk image having a pixel value obtained by multiplying the crosstalk ratio by the pixel value of the first L image.

5. The display apparatus of claim 2 , wherein the processor is further configured to:

obtain a third R image in which the first R image and the second R image are synthesized and obtain the R crosstalk image having a pixel value by multiplying a crosstalk ratio by the pixel value of the synthesized third R image; and

obtain a third L image in which the first L image and the second L image are synthesized and obtain the L crosstalk image having a pixel value obtained by multiplying the crosstalk ratio by the pixel value of the synthesized third L image.

6. The display apparatus of claim 1 , wherein the processor is further configured to:

obtain the L image by inputting the LF image representing the left viewpoint to a first factorization model trained to output the L image corresponding to a number of the first display panel and the second display panel; and

obtain the R image by inputting the LF image representing the right viewpoint to a second factorization model trained to output the R image corresponding to a number of the first display panel and the second display panel.

7. The display apparatus of claim 6 ,

wherein the first factorization model is an artificial intelligence model trained until a loss function based on an LF image representing a left viewpoint and an LF image restored through the L image becomes less than or equal to a predetermined value, and

wherein the second factorization model is an artificial intelligence model trained until a loss function based on an LF image representing a right point of view and an LF image restored through the R image becomes less than or equal to a predetermined value.

8. The display apparatus of claim 6 , wherein each of the first factorization model and the second factorization model is one of a deep neural network (DNN) model, a non-negative tensor factorization (NTF) model, or a non-negative matrix factorization (NMF) model.

9. The display apparatus of claim 1 , wherein the lens array comprises a unit lens disposed on an even number of sub-pixels of the first display panel.

10. The display apparatus of claim 1 , wherein the processor is further configured to:

based on the LF image which are classified according to a viewpoint at which the plurality of LF images are captured, obtain the L image, an M image, and the R image corresponding to the viewpoint;

correct the second L image based on a first M image and the first R image for driving the first display panel among the M images;

correct the second R image based on the first M image and the first L image;

based on the first L image and the first R image, correct a second M image for driving the second display panel among the M images; and

display the stereoscopic image by driving the display by time-multiplexing based on the L image which comprises the corrected second L image, the M image which comprises the corrected second M image, and the R image which comprises the corrected second R image.

11. A controlling method of a display apparatus, the controlling method comprising:

based on a plurality of light field (LF) images, obtaining a left (L) image and a right (R) image to drive a display comprising a first display panel comprising a plurality of sub-pixels, a lens array disposed on the first display panel, and a second display panel disposed on the lens array by time-multiplexing;

correcting a second L image to drive the second display panel among L images based on a first R image to drive the first display panel among R images, and correct a second R image to drive the second display panel among the R images based on a first L image to drive the first display panel among the L images; and

displaying a stereoscopic image by driving the display by time-multiplexing based on the L image which comprises the corrected second L image and the R image which comprises the corrected second R image,

wherein the lens array comprises a plurality of unit lens and is disposed on the first display panel such that respectively unit lens is arranged on a predetermined number of first sub-pixels, and

wherein the lens array disposed on the first display panel such that respectively unit lens is arranged on a predetermined number of the first sub-pixels, and

wherein the obtaining a left (L) image and a right (R) image comprises:

based on information of a viewpoint at which a plurality of LF images are captured, extracting an LF image representing a left viewpoint and an LF image representing a right viewpoint among the plurality of LF images,

based on the LF image representing the left viewpoint obtaining a left (L) images respectively corresponding to the first display panel and the second display panel, and

based on the LF image representing the right viewpoint obtaining a right (R) images respectively corresponding to the first display panel and the second display panel.

12. The method of claim 11 , wherein the correcting comprises:

obtaining an R crosstalk image based on the first R image and an L crosstalk image based on the first L image; and

correcting the second L image based on a pixel value of the R crosstalk image, and correcting the second R image based on a pixel value of the L crosstalk image.

13. The method of claim 12 , wherein the correcting comprises:

correcting the second L image to have a pixel value reduced as much as a pixel value of the R crosstalk image with respect to a pixel of a same position; and

correcting the second R image to have a pixel value reduced as much as a pixel value of the L crosstalk image with respect to a pixel of a same position.

14. The method of claim 12 , wherein the obtaining the R crosstalk image and the L crosstalk image comprises:

obtaining the R crosstalk image having a pixel value obtained by multiplying a crosstalk ratio by the pixel value of the first R image; and

obtaining the L crosstalk image having a pixel value obtained by multiplying the crosstalk ratio by the pixel value of the first L image.

15. The method of claim 12 , wherein the obtaining the R crosstalk image and the L crosstalk image comprises:

obtaining a third R image in which the first R image and the second R image are synthesized and obtain the R crosstalk image having a pixel value by multiplying a crosstalk ratio by the pixel value of the synthesized third R image; and

obtaining a third L image in which the first L image and the second L image are synthesized and obtain the L crosstalk image having a pixel value obtained by multiplying the crosstalk ratio by the pixel value of the synthesized third L image.

16. The method of claim 12 , wherein the plurality of LF images representing the left viewpoint are restored based on the first L image and the second L image included in the L image.

17. The method of claim 16 , wherein the restored LF image is an LF image that is viewed at an external specific viewpoint when a first display panel displays the first L image and a second display panel displays the second L image through an actual image or a virtual simulation.

18. The method of claim 17 , wherein a shifting parameter of an image displayed on each of the first display panel and the second display panel is mapped for each viewpoint.