Imaging apparatus, unmanned moving object, imaging method, system, and program
An imaging apparatus, an unmanned moving object, an imaging method, a system, and a program capable of favorably compositing a telephoto image group even in a case where an overlapping region between images of the telephoto image group is small, and accurately compositing a telephoto image regardless of a subject (scene) of a wide angle image are provided. An imaging apparatus ( 100 ) includes an imaging optical system, a directional sensor, a wide dynamic range image generation part ( 302 ) that generates a wide dynamic range wide angle image obtained by enlarging a dynamic range of a wide angle image, an image acquisition part ( 2 a ) that acquires a wide dynamic range wide angle image group and a telephoto image group configured with a telephoto image, a composition information acquisition part ( 2 b ) that acquires composition information to be used for compositing the telephoto image group by analyzing the acquired wide dynamic range wide angle image group, and a composite image generation part ( 2 c ) that generates an image in which the telephoto image group is composited, based on the composition information, information related to focal lengths of the wide angle optical system and the telephoto optical system, and the telephoto image group.
1. An imaging apparatus comprising:
an imaging optical system including a wide angle optical system and a telephoto optical system having a common optical axis;
a directional sensor that includes a plurality of pixels configured with photoelectric conversion elements arranged in two dimensions, the directional sensor including the plurality of pixels selectively receiving luminous flux incident through each of the wide angle optical system and the telephoto optical system by pupil separation; and
at least one processor configured to:
generate, based on an image signal from the directional sensor, a wide dynamic range wide angle image obtained by enlarging a dynamic range of a wide angle image captured through the wide angle optical system;
acquire a wide dynamic range wide angle image group configured with the generated wide dynamic range wide angle image, and a telephoto image group which is captured at the same time as the wide dynamic range wide angle image group and is configured with a telephoto image which is the image signal from the directional sensor and is captured through the telephoto optical system, wherein a subject is imaged in the wide dynamic range wide angle image group and the telephoto image group while an imaging position is changed;
acquire composition information to be used for compositing the telephoto image group by analyzing the acquired wide dynamic range wide angle image group; and
generate an image in which the telephoto image group is composited, based on the composition information, information related to focal lengths of the wide angle optical system and the telephoto optical system, and the telephoto image group.
2. The imaging apparatus according to claim 1 ,
wherein the at least one processor detects a feature point of the wide dynamic range wide angle image by analyzing the wide dynamic range wide angle image group.
3. The imaging apparatus according to claim 1 ,
wherein the at least one processor estimates the imaging position and an attitude in a case of capturing the wide dynamic range wide angle image and a solid shape of the subject using a structure from motion method or a simultaneous localization and mapping method.
4. The imaging apparatus according to claim 1 , further comprising:
the at least one processor is further configured to determine whether or not to execute generation of the wide dynamic range wide angle image in accordance with a width of an intensity of light of the wide angle image,
acquire a wide angle image group configured with the wide angle image and the telephoto image group in a case where the wide dynamic range wide angle image is not generated, and
acquire the composition information to be used for compositing the telephoto image group by analyzing the wide angle image group.
5. The imaging apparatus according to claim 4 ,
wherein the at least one processor performs determination in accordance with a width of an intensity of light of a region of the wide angle image corresponding to a region to be analyzed in the wide dynamic range wide angle image.
6. The imaging apparatus according to claim 1 ,
wherein the at least one processor generates the wide dynamic range wide angle image based on the wide angle image obtained by performing imaging at an exposure smaller than an appropriate exposure.
7. The imaging apparatus according to claim 1 ,
wherein the at least one processor generates the wide dynamic range wide angle image based on two wide angle images of different exposures.
8. The imaging apparatus according to claim 7 ,
wherein the directional sensor receives luminous flux corresponding to a plurality of the wide angle images of different exposures at the same time.
9. An unmanned moving object comprising:
the imaging apparatus according to claim 7 ; and
at least one processor configured to control movement,
wherein the at least one processor decreases a movement speed in a case of acquiring two wide angle images of different exposures.
10. An unmanned moving object comprising:
the imaging apparatus according to claim 1 ; and
at least one processor configured to control movement.
11. An imaging method of an imaging apparatus including an imaging optical system including a wide angle optical system and a telephoto optical system having a common optical axis, and a directional sensor that includes a plurality of pixels configured with photoelectric conversion elements arranged in two dimensions, the directional sensor including the plurality of pixels selectively receiving luminous flux incident through each of the wide angle optical system and the telephoto optical system by pupil separation, the imaging method comprising:
generating, based on an image signal from the directional sensor, a wide dynamic range wide angle image obtained by enlarging a dynamic range of a wide angle image captured through the wide angle optical system;
acquiring a wide dynamic range wide angle image group configured with the wide dynamic range wide angle image, and a telephoto image group which is captured at the same time as the wide dynamic range wide angle image group and is configured with a telephoto image which is the image signal from the directional sensor and is captured through the telephoto optical system, a subject being imaged in the wide dynamic range wide angle image group and the telephoto image group while an imaging position is changed;
acquiring composition information to be used for compositing the telephoto image group by analyzing the acquired wide dynamic range wide angle image group; and
generating an image in which the telephoto image group is composited, based on the composition information, information related to focal lengths of the wide angle optical system and the telephoto optical system, and the telephoto image group.
12. The imaging method according to claim 11 ,
wherein in acquiring the composition information, a feature point of the wide dynamic range wide angle image is detected by analyzing the wide dynamic range wide angle image group.
13. The imaging method according to claim 11 ,
wherein in acquiring the composition information, the imaging position and an attitude in a case of capturing the wide dynamic range wide angle image and a solid shape of the subject are estimated using a structure from motion method or a simultaneous localization and mapping method.
14. The imaging method according to claim 11 , further comprising:
determining whether or not to execute generation of the wide dynamic range wide angle image in generating the wide dynamic range wide angle image, in accordance with a width of an intensity of light of the wide angle image,
wherein in acquiring the image, a wide angle image group configured with the wide angle image and the telephoto image group are acquired in a case where the wide dynamic range wide angle image is not generated, and
in acquiring the composition information, the composition information to be used for compositing the telephoto image group is acquired by analyzing the wide angle image group.
15. A system comprising an imaging apparatus and an image generation apparatus,
wherein the imaging apparatus includes:
an imaging optical system including a wide angle optical system and a telephoto optical system having a common optical axis;
a directional sensor having a plurality of pixels configured with photoelectric conversion elements arranged in two dimensions, the directional sensor including the plurality of pixels selectively receiving luminous flux incident through each of the wide angle optical system and the telephoto optical system by pupil separation; and
a camera-side micom configured to generate, based on an image signal from the directional sensor, a wide dynamic range wide angle image obtained by enlarging a dynamic range of a wide angle image captured through the wide angle optical system, and
the image generation apparatus includes at least one processor configured to:
acquire a wide dynamic range wide angle image group configured with the wide dynamic range wide angle image generated by the camera-side micom, and a telephoto image group which is captured at the same time as the wide dynamic range wide angle image group and is configured with a telephoto image which is the image signal from the directional sensor and is captured through the telephoto optical system, a subject being imaged in the wide dynamic range wide angle image group and the telephoto image group while an imaging position is changed;
acquire composition information to be used for compositing the telephoto image group by analyzing the acquired wide dynamic range wide angle image group; and
generate an image in which the telephoto image group is composited, based on the composition information, information related to focal lengths of the wide angle optical system and the telephoto optical system, and the telephoto image group.
16. A non-transitory computer-readable recording medium causing a computer to execute an imaging step of an imaging apparatus in a case where an instruction stored in the recording medium is read out by the computer, the imaging apparatus including an imaging optical system including a wide angle optical system and a telephoto optical system having a common optical axis, and a directional sensor that includes a plurality of pixels configured with photoelectric conversion elements arranged in two dimensions, the directional sensor including the plurality of pixels selectively receiving luminous flux incident through each of the wide angle optical system and the telephoto optical system by pupil separation, the imaging step comprising:
generating, based on an image signal from the directional sensor, a wide dynamic range wide angle image obtained by enlarging a dynamic range of a wide angle image captured through the wide angle optical system;
acquiring a wide dynamic range wide angle image group configured with the wide dynamic range wide angle image, and a telephoto image group which is captured at the same time as the wide dynamic range wide angle image group and is configured with a telephoto image which is the image signal from the directional sensor and is captured through the telephoto optical system, a subject being imaged in the wide dynamic range wide angle image group and the telephoto image group while an imaging position is changed;
acquiring composition information to be used for compositing the telephoto image group by analyzing the acquired wide dynamic range wide angle image group; and
generating an image in which the telephoto image group is composited, based on the composition information, information related to focal lengths of the wide angle optical system and the telephoto optical system, and the telephoto image group.