Image processing device, image processing method, and storage medium

1 . An image processing device comprising:

an image acquisition device configured to acquire an image containing a subject via a lens unit;

a memory storing instructions; and

at least one processor executing the instructions causing the image processing device to:

acquire distance information indicating a distance to the subject;

generate auxiliary data related to the distance information;

generate a data stream conforming to a transmission standard of a serial digital interface (SDI) in which the image, the distance information, and the auxiliary data are superimposed; and

output the data stream in conformity with the transmission standard of the SDI to outside the image processing device,

wherein the at least one processor executing the instructions further causes the image processing device to superimpose the distance information on a data stream of luminance or a data stream of color,

wherein an image format for transmission is YCbCr422,

wherein (1) the distance information is superimposed in Cr and a red gradation of the distance information is displayed on an external display, or (2) the distance information is superimposed in Cb and a blue gradation of the distance information is displayed on the external display, and

wherein in the image format for transmission Cb and Cr are alternately superimposed on C data for each word.

2 . The image processing device according to claim 1 ,

wherein the image acquisition device includes an image sensor in which a plurality of photoelectric conversion units receiving light fluxes passing through different pupil regions of the lens unit are arranged for each pixel, and

wherein the distance information is acquired based on a phase difference between outputs of the plurality of photoelectric conversion units.

3 . The image processing device according to claim 2 , wherein the image is acquired by adding the outputs of the plurality of photoelectric conversion units.

4 . The image processing device according to claim 1 , wherein the at least one processor executing the instructions further causing the image processing device to divide and superimpose the distance information on a plurality of data streams.

5 . The image processing device according to claim 1 , wherein the at least one processor executing the instructions further causing the image processing device to superimpose the distance information on a data stream of an alpha channel.

6 . The image processing device according to claim 1 , wherein the at least one processor executing the instructions further causing the image processing device to select whether the distance information is divided and superimposed on a plurality of data streams or the distance information is superimposed on one data stream.

7 . The image processing device according to claim 6 , wherein the at least one processor executing the instructions further causing the image processing device to select whether the distance information is superimposed on a data stream of luminance or the distance information is superimposed on a data stream of color.

8 . The image processing device according to claim 6 , wherein the at least one processor executing the instructions further causing the image processing device to select whether the distance information is superimposed on a data steam of an alpha channel.

9 . The image processing device according to claim 1 , wherein the at least one processor executing the instructions further causing the image processing device to select a number of effective bits of the distance information.

10 . The image processing device according to claim 1 , wherein the at least one processor executing the instructions further causing the image processing device to superimpose high-order bits of the distance information in a number of effective bits and superimpose 0 on low-order bits.

11 . The image processing device according to claim 1 , wherein the auxiliary data includes data regarding setting data or a state of the image processing device.

12 . The image processing device according to claim 1 , wherein the auxiliary data includes lens information regarding the lens unit.

13 . The image processing device according to claim 12 , wherein the lens information includes at least one of focal distance, a focusing position, and diaphragm information.

14 . The image processing device according to claim 1 , wherein the auxiliary data includes data for synchronizing the image and the distance information.

15 . The image processing device according to claim 1 , wherein the auxiliary data includes data indicating an attribute of the distance information.

16 . The image processing device according to claim 1 , wherein the auxiliary data includes information regarding a data structure of the data stream.

17 . The image processing device according to claim 1 , wherein the data stream includes still image data.

18 . The image processing device according to claim 1 , wherein the at least one processor executing the instructions further causing the image processing device to perform the superimposition by multiplexing or mixing.

19 . The image processing device according to claim 1 , wherein

when the distance information exceeds a threshold, the distance information is superimposed in Cb and a fixed value is superimposed in Cr, and when the distance information is less than the threshold, the distance information is superimposed in Cr and the fixed value is superimposed in Cb, or

when the distance information exceeds a threshold, the distance information is superimposed in Cr and a fixed value is superimposed in Cb, and when the distance information is less than the threshold, the distance information is superimposed in Cb and the fixed value is superimposed in Cr.

20 . The image processing device according to claim 1 , wherein the distance information is superimposed at positions of both Y data and C data, and both luminance and color changed in accordance with a value of the distance information are displayed on the external display.

21 . A method of controlling an image processing device, the method comprising:

acquiring an image containing a subject via a lens unit;

acquiring distance information indicating a distance to the subject;

generating auxiliary data related to the distance information;

generating a data stream conforming to a transmission standard of a serial digital interface (SDI) in which the image, the distance information, and the auxiliary data are superimposed; and

outputting the data stream in conformity with the transmission standard of the SDI to outside the image processing device,

wherein the distance information is superimposed on a data stream of luminance or a data stream of color,

wherein an image format for transmission is YCbCr422,

wherein (1) the distance information is superimposed in Cr and a red gradation of the distance information is displayed on an external display, or (2) the distance information is superimposed in Cb and a blue gradation of the distance information is displayed on the external display, and

wherein in the image format for transmission Cb and Cr are alternately superimposed on C data for each word.

22 . A non-transitory computer-readable storage medium configured to store a computer program comprising instructions for executing a method of controlling an image processing device, the method comprising:

acquiring an image containing a subject via a lens unit;

acquiring distance information indicating a distance to the subject;

generating auxiliary data related to the distance information;

generating a data stream conforming to a transmission standard of a serial digital interface (SDI) in which the image, the distance information, and the auxiliary data are superimposed; and

outputting the data stream in conformity with the transmission standard of the SDI to outside the image processing device,

wherein the distance information is superimposed on a data stream of luminance or a data stream of color,

wherein an image format for transmission is YCbCr422,

wherein (1) the distance information is superimposed in Cr and a red gradation of the distance information is displayed on an external display, or (2) the distance information is superimposed in Cb and a blue gradation of the distance information is displayed on the external display, and

wherein in the image format for transmission Cb and Cr are alternately superimposed on C data for each word.

23 . An image processing device comprising:

an image acquisition device configured to acquire an image containing a subject via a lens unit;

a memory storing instructions; and

at least one processor executing the instructions causing the image processing device to:

acquire distance information indicating a distance to the subject;

generate auxiliary data related to the distance information;

generate a data stream conforming to a transmission standard of a serial digital interface (SDI) in which the image, the distance information, and the auxiliary data are superimposed; and

output the data stream in conformity with the transmission standard of the SDI to outside the image processing device,

wherein the at least one processor executing the instructions further causes the image processing device to superimpose high-order bits of the distance information in a number of effective bits and superimpose 0 on low-order bits,

wherein an image format for transmission is YCbCr422, and

wherein in the image format for transmission Cb and Cr are alternately superimposed on C data for each word.

24 . A method of controlling an image processing device, the method comprising:

acquiring an image containing a subject via a lens unit;

acquiring distance information indicating a distance to the subject;

generating auxiliary data related to the distance information;

generating a data stream conforming to a transmission standard of a serial digital interface (SDI) in which the image, the distance information, and the auxiliary data are superimposed;

outputting the data stream in conformity with the transmission standard of the SDI to outside the image processing device; and

superimposing high-order bits of the distance information in a number of effective bits and superimposing 0 on low-order bits,

wherein an image format for transmission is YCbCr422, and

wherein in the image format for transmission Cb and Cr are alternately superimposed on C data for each word.