Image generation device and image generation method for generating a composite image
An embodiment of the present disclosure relates to an image generation device including an image acquisition unit, a position recognition unit, a boundary line setting unit, and an image compositing unit. The image acquisition unit is mounted to a vehicle, and configured to acquire a plurality of captured images in which the periphery of the vehicle is imaged by a plurality of imaging units having an overlap region where imaging regions thereof partially overlap with each other. The position recognition unit is configured to recognize positions of specific obstacles which are one or more obstacles positioned in the overlap region. The boundary line setting unit is configured to set the boundary line so that each of the specific obstacles is contained in the image captured by the imaging unit which is closest in distance to the specific obstacle, among the plurality of imaging units. The image compositing unit is configured to generate a composite image in which the plurality of captured images are combined, using the boundary line as a boundary when combining the plurality of captured images.
1. An image generation device comprising:
an image acquisition unit mounted to a vehicle, and configured to acquire a plurality of captured images in which a periphery of the vehicle is imaged by a plurality of imaging units having an overlap region where imaging regions thereof partially overlap with each other;
a position recognition unit configured to recognize positions of specific obstacles which are one or more obstacles positioned in the overlap region;
a boundary line setting unit configured to set a boundary line so that each of the specific obstacles is contained in an image captured by an imaging unit, among the plurality of imaging units, which is closest in distance to a respective specific obstacle; and
an image compositing unit configured to generate a composite image in which the plurality of captured images are combined, using the boundary line as a boundary when combining the plurality of captured images, wherein
the boundary line setting unit temporarily sets a plurality of boundary line candidates within the overlap region while avoiding the specific obstacles, and sets, as the boundary line, the boundary line candidate which is closest to a preliminarily set reference position among the plurality of boundary line candidates.
2. The image generation device according to claim 1 , wherein
the distance to each of the specific obstacles is a distance to a location where the specific obstacles contact the ground, and
the boundary line setting unit is configured to set the boundary line so that the respective specific obstacle is contained in the image captured by the imaging unit which is closest in distance to a ground-contact location of the respective specific obstacle.
3. The image generation device according to claim 1 , wherein
the image compositing unit is configured to generate a bird's eye view image in which the periphery of the vehicle is as seen from above, as the composite image, and
the image generation device further includes an image replacement unit configured to replace the specific obstacles to be enlarged in the bird's eye view image with other images.
4. The image generation device according to claim 3 , further comprising:
an image complementation unit configured to complement regions produced by replacing the specific obstacles with other images, using the captured images of the overlap region captured by imaging units, of the plurality of imaging units, which are not imaging units closest in distance to the respective specific obstacles.
5. An image generation device for generating a display image to be displayed on a display device being arranged in a vehicle compartment from images captured by a plurality of cameras which are arranged at different positions of a vehicle, the image generation device comprising:
a control unit which includes a memory and a CPU, wherein
the images captured by the plurality of cameras have an overlap region where they partially overlap with each other;
the control unit recognizes positions of one or more obstacles included in the overlap region of the images captured by the plurality of cameras, which are stored in the memory, based on processing by the CPU;
the control unit sets a boundary line so that each of the one or more obstacles recognized based on the processing by the CPU is contained in an image captured by a camera, among the plurality of cameras, which is closest in distance to a respective obstacle; and
the control unit generates a composite image in which the plurality of captured images are combined, using the boundary line as a boundary when combining the plurality of captured images, wherein
the control unit temporarily sets a plurality of boundary line candidates within the overlap region while avoiding the one or more obstacles, and sets, as the boundary line, the boundary line candidate which is closest to a preliminarily set reference boundary line among the plurality of boundary line candidates.
6. The image generation device according to claim 5 , wherein
the distance to each of the one or more obstacles is a distance to a location where the one or more obstacles contact the ground, and
the control unit sets the boundary line so that each of the one or more obstacles is contained in the image captured by the camera which is closest in distance to a ground-contact location of the respective obstacle.
7. The image generation device according to claim 5 , wherein
the control unit is configured to generate a bird's eye view image in which a periphery of the vehicle is as seen from above, as the composite image, and
the control unit replaces the one or more obstacles displayed in an enlarged manner in the bird's eye view image with other images to generate the composite image.
8. The image generation device according to claim 7 , wherein
the control unit complements shade regions produced after replacing the one or more obstacles with other images, using the captured images of the overlap region captured by cameras, among the plurality of cameras, which are not cameras closest in distance to the respective one or more obstacles.
9. An image generation method to be executed by a control unit which is mounted to a vehicle, comprising:
acquiring a plurality of captured images in which a periphery of the vehicle is imaged by a plurality of imaging units having an overlap region where imaging regions thereof partially overlap with each other;
recognizing each of positions of specific obstacles which are one or more obstacles positioned in the overlap region;
setting a boundary line so that each of the specific obstacles is contained in an image captured by an imaging unit, among the plurality of imaging units, which is closest in distance to a respective specific obstacle;
generating a composite image in which the plurality of captured images are combined, using the boundary line as a boundary when combining the plurality of captured images;
temporarily setting a plurality of boundary line candidates within the overlap region while avoiding the specific obstacles; and
setting, as the boundary line, the boundary line candidate which is closest to a preliminarily set reference position among the plurality of boundary line candidates.
10. An image generation device comprising:
an image acquisition unit mounted to a vehicle, and configured to acquire a plurality of captured images in which a periphery of the vehicle is imaged by a plurality of imaging units having an overlap region where imaging regions thereof partially overlap with each other;
a position recognition unit configured to recognize positions of specific obstacles which are one or more obstacles positioned in the overlap region;
a boundary line setting unit configured to set a boundary line so that each of the specific obstacles is contained in an image captured by an imaging unit, among the plurality of imaging units, which is closest in distance to a respective specific obstacle;
an image compositing unit configured to generate a composite image in which the plurality of captured images are combined, using the boundary line as a boundary when combining the plurality of captured images, the image compositing unit configured to generate a bird's eye view image in which the periphery of the vehicle is as seen from above, as the composite image;
an image replacement unit configured to replace the specific obstacles to be enlarged in the bird's eye view image with other images; and
an image complementation unit configured to complement regions produced by replacing the specific obstacles with other images, using the captured images of the overlap region captured by imaging units, among the plurality of imaging units, which are not the imaging units closest in distance to respective specific obstacles.
11. The image generation device according to claim 10 , wherein
the boundary line setting unit temporarily sets a plurality of boundary line candidates within the overlap region while avoiding the specific obstacles, and sets, as the boundary line, the boundary line candidate which is closest to a preliminarily set reference position among the plurality of boundary line candidates.
12. The image generation device according to claim 10 , wherein
the distance to each of the specific obstacles is a distance to a location where the specific obstacles contact the ground, and
the boundary line setting unit is configured to set the boundary line so that the respective specific obstacle is contained in the image captured by the imaging unit which is closest in distance to a ground-contact location of the respective specific obstacle.
13. An image generation device for generating a display image to be displayed on a display device being arranged in a vehicle compartment from images captured by a plurality of cameras which are arranged at different positions of a vehicle, the image generation device comprising:
a control unit which includes a memory and a CPU, wherein
the images captured by the plurality of cameras have an overlap region where they partially overlap with each other;
the control unit recognizes positions of one or more obstacles included in the overlap region of the images captured by the plurality of cameras, which are stored in the memory, based on processing by the CPU;
the control unit sets a boundary line so that each of the one or more obstacles recognized based on the processing by the CPU is contained in an image captured by a camera, among the plurality of cameras, which is closest in distance to a respective obstacle; and
the control unit generates a composite image in which the plurality of captured images are combined, using the boundary line as a boundary when combining the plurality of captured images, wherein
the control unit is configured to generate a bird's eye view image in which a periphery of the vehicle is as seen from above, as the composite image,
the control unit replaces the one or more obstacles displayed in an enlarged manner in the bird's eye view image with other images to generate the composite image, and
the control unit complements shade regions produced after replacing the one or more obstacles with other images, using the captured images of the overlap region captured by cameras, among the plurality of cameras, which are not cameras closest in distance to the respective obstacles.
14. The image generation device according to claim 13 , wherein
the control unit temporarily sets a plurality of boundary line candidates within the overlap region while avoiding the one or more obstacles, and sets, as the boundary line, the boundary line candidate which is closest to a preliminarily set reference boundary line among the plurality of boundary line candidates.
15. The image generation device according to claim 13 , wherein
the distance to each of the one or more obstacles is a distance to a location where the one or more obstacles contact the ground, and
the control unit sets the boundary line so that each of the one or more obstacles is contained in the image captured by the camera which is closest in distance to a ground-contact location of the respective obstacle.