Image processing apparatus, control method thereof, and non-transitory computer-readable storage medium

1 . An image processing apparatus comprising:

at least one processor and at least one memory, the at least one processor and at least one memory being configured to function as a plurality of units comprising:

(A) an acquisition unit configured to acquire information relating to position and orientation of an image capturing apparatus configured to capture an image of a subject with an image displayed on a display apparatus as a background;

(B) a calculation unit configured to calculate a correction coefficient corresponding to each display element included in the display apparatus, based on display position of the display apparatus and the information acquired by the acquisition unit;

(D) a correction unit configured to correct, by the correction coefficient calculated by the calculation unit, a pixel value of each pixel in an image displayed on the display apparatus; and

(E) a control unit configured to (1) control the correction unit, (2) output an image being corrected to the display apparatus, and (3) generate and output correction information relating to exposure with respect to the image capturing apparatus,

wherein the control unit includes a determination unit configured to determine whether or not the image corrected by the correction unit includes a pixel with a value exceeding a maximum value that can be input to the display apparatus,

wherein when a determination result of the determination unit indicates that the image being corrected does not include any pixel with a value exceeding the maximum value that can be input to the display apparatus, the control unit (1) outputs the image being corrected to the display apparatus and (2) maintains an initial exposure amount with respect to the image capturing apparatus, and

wherein when the determination result of the determination unit indicates that the image being corrected includes a pixel with a value exceeding the maximum value that can be input to the display apparatus, the control unit (1) outputs, to the display apparatus, an image obtained by performing re-correction on the image being corrected such that the pixel value becomes equal to or lower than the maximum value, and (2) generates and outputs, to the image capturing apparatus, correction information for increasing the exposure amount based on the re-correction.

2 . The image processing apparatus according to claim 1 , wherein the display apparatus includes a display screen that is formed by arranging one or more display panels each of which is constituted by a display element having a brightness characteristic depending on an angle.

3 . The image processing apparatus according to claim 2 , wherein the calculation unit obtains, based on the display position of the display apparatus and the information acquired by the acquisition unit, a vector oriented from each of the display elements toward the image capturing apparatus, and calculates the correction coefficient that depends on the characteristic for each display element.

4 . The image processing apparatus according to claim 2 , wherein the display element is a Light Emitting Diode (LED) element, and

wherein the display panel is an LED panel.

5 . The image processing apparatus according to claim 1 , wherein the control unit further generates information indicating an intensity of illumination for an external illumination apparatus and outputs the information to the external illumination apparatus.

6 . The image processing apparatus according to claim 1 , wherein the acquisition unit further acquires information relating to an angle-of-view from the image capturing apparatus,

wherein the calculation unit includes a determination unit configured to determine, based on (1) the position of the display apparatus, (2) the position and orientation of the image capturing apparatus, and (3) the information relating to the angle-of-view, an area included in the angle-of-view on a display screen as a correction area, and

wherein the calculation unit calculates the correction coefficient in the correction area.

7 . The image processing apparatus according to claim 6 , wherein the plurality of units further comprises a unit configured to acquire information representing individual differences relating to light emission of each display panel included in the display screen of the display apparatus, and

wherein the correction unit further corrects the image being corrected, based on the information representing the individual differences for each display panel.

8 . The image processing apparatus according to claim 7 , wherein the information representing the individual differences is information indicating a ratio of brightness to a reference value of each of red (R), green (G), and blue (B) of each display panel.

9 . The image processing apparatus according to claim 1 , wherein the correction unit generates and corrects an image displayed on the display apparatus by rendering based on Computer Graphics (CG) data stored in a storage unit.

10 . The image processing apparatus according to claim 1 , further comprising an output unit configured to output the image corrected by the correction unit to the display apparatus.

11 . A control method for an image processing apparatus, the control method comprising:

acquiring information relating to position and orientation of an image capturing apparatus configured to capture an image of a subject with an image displayed on a display apparatus as a background;

calculating a correction coefficient corresponding to each display element included in the display apparatus, based on display position of the display apparatus and the information being acquired;

correcting, by the correction coefficient being calculated, a pixel value of each pixel in an image displayed on the display apparatus;

determining whether or not an image corrected by the correcting includes a pixel with a value exceeding a maximum value that can be input to the display apparatus;

when it is determined that the image being corrected does not include any pixel with a value exceeding a maximum value that can be input to the display apparatus, (1) outputting the image being corrected to the display apparatus and (2) maintaining an initial exposure amount with respect to the image capturing apparatus; and

when it is determined that the image being corrected includes a pixel with a value exceeding the maximum value that can be input to the display apparatus, (1) outputting, to the display apparatus, an image obtained by performing re-correction on the image being corrected such that the pixel value becomes equal to or lower than the maximum value, and (2) generating and outputting, to the image capturing apparatus, correction information for increasing the exposure amount based on the re-correction.

12 . A non-transitory computer-readable storage medium storing a computer program configured to, when the computer program is read and executed by a computer, cause the computer to:

acquire information relating to position and orientation of an image capturing apparatus configured to capture an image of a subject with an image displayed on a display apparatus as a background;

calculate a correction coefficient corresponding to each display element included in the display apparatus, based on the display position of the display apparatus and the information being acquired;

correct, by the correction coefficient being calculated, a pixel value of each pixel in an image displayed on the display apparatus;

determine whether or not an image corrected by the correcting includes a pixel with a value exceeding a maximum value that can be input to the display apparatus;

when it is determined that the image being corrected does not include any pixel with a value exceeding a maximum value that can be input to the display apparatus, (1) output the image being corrected to the display apparatus and (2) maintain an initial exposure amount with respect to the image capturing apparatus; and

when it is determined that the image being corrected includes a pixel with a value exceeding the maximum value that can be input to the display apparatus, (1) output, to the display apparatus, an image obtained by performing re-correction on the image being corrected such that the pixel value becomes equal to or lower than the maximum value, and (2) generate and output, to the image capturing apparatus, correction information for increasing the exposure amount based on the re-correction.