Image processing device and image processing method

1. An image processing device comprising a processor programmed to:

acquire a captured image where an image of an environment of a vehicle is captured by an image-capturing device mounted on the vehicle, the image-capturing device including an image sensor;

acquire, from a detection device mounted on the vehicle, information regarding a detected position of an obstacle that exists in an environment of the vehicle;

generate a virtual viewpoint image where an environment of the vehicle is viewed from a virtual viewpoint, based on the acquired captured image;

composite a notification image that provides notification of existence of the obstacle at the detected position with the virtual viewpoint image and composite a vehicle image with the virtual viewpoint image; and

composite the vehicle image with the virtual viewpoint image composited with the notification image, while overlapping the vehicle image and the notification image with each other, to form a composited vehicle image, where (i) at least a part of an overlapped part in the composited vehicle image between the vehicle and the obstacle is transparent, (ii) the vehicle and the notification overlap each other in the composited vehicle image, and (iii) in the overlapped part of the composited vehicle image, the vehicle image is transparent in order to not hide the notification image so that the notification image is viewed through the transparent vehicle image,

wherein acquiring the information regarding the detected position of the obstacle that exists in the environment of the vehicle includes (i) segmenting a detection range in the environment into a plurality of regions defined by intersecting longitudinal areas and latitudinal areas, (ii) determining the longitudinal area where the obstacle exists in the detection range, (iii) determining the latitudinal area where the obstacle exists in the detection range, and (iv) identifying a target region where the obstacle exists in the detection range based on where the determined longitudinal area and the determined latitudinal area intersect.

2. The image processing device according to claim 1 , wherein the processor is further programmed to cause at least a part of the notification image to be transparent and composite it with the virtual viewpoint image.

3. The image processing device according to claim 1 , wherein the detection range includes at least four longitudinal areas, at least four latitudinal areas and at least sixteen target regions.

4. The image processing device according to claim 1 , wherein the processor is further programmed to composite therewith the vehicle image where a profile line therein is not transparent.

5. The image processing device according to claim 1 , wherein the processor is further programmed to composite therewith the vehicle image that is rendered with a plurality of profile lines with different brightness.

6. The image processing device according to claim 1 , wherein the processor is further programmed to composite therewith the notification image that is three-dimensionally displayed.

7. The image processing device according to claim 1 , wherein the processor is further programmed to:

acquire the information regarding the detected position from the detection device that detects the obstacle; and

composite a range image that indicates the detection range of the detection device with the virtual viewpoint image.

8. The image processing device according to claim 7 , wherein the processor is further programmed to display an outer frame of a range image that indicates the detection range in a flare-like shape and composite it with the virtual viewpoint image.

9. The image processing device according to claim 1 , wherein the processor is further programmed to generate the virtual viewpoint image where the vehicle is viewed from an upper side in back of the vehicle.

10. The image processing device according to claim 1 , wherein the processor is further programmed to emphatically display the notification image depending on the detected position.

11. An image processing device comprising a processor programmed to:

acquire a captured image where an image of an environment of a passenger car is captured by an image-capturing device mounted on the passenger-car, the image-capturing device including an image sensor;

generate a virtual viewpoint image where an environment of the passenger car is viewed from a virtual viewpoint, based on the acquired captured image; and

composite a passenger-car image with the virtual viewpoint image and a notification image to form a composited passenger-car image, where (i) at least a part of the passenger car therein is transparent with the virtual viewpoint image, (ii) the passenger-car and the notification overlap each other in the composited passenger-car image, and (iii) in a part of the composited passenger-car image where the passenger-car and the notification overlap each other, the passenger-car image is transparent in order to not hide the notification image so that the notification image is viewed through the transparent passenger-car image,

wherein compositing the passenger-car image with the virtual viewpoint image and the notification image includes (i) acquiring information regarding a detected position of an obstacle that exists in an environment of the passenger-car, (ii) segmenting a detection range in the environment into a plurality of regions defined by intersecting longitudinal areas and latitudinal areas, (iii) determining the longitudinal area where the obstacle exists in the detection range, (iv) determining the latitudinal area where the obstacle exists in the detection range, and (v) identifying a target region where the obstacle exists in the detection range based on where the determined longitudinal area and the determined latitudinal area intersect.

12. The image processing device according to claim 11 , wherein the processor is further programmed to composite the vehicle image where a profile line therein is not transparent with the virtual viewpoint image.

13. An image processing method comprising:

acquiring a captured image where an image of an environment of a vehicle is captured by an image-capturing device mounted on the vehicle, the image-capturing device including an image sensor;

acquiring, from a detection device mounted on the vehicle, information regarding a detected position of an obstacle that exists in an environment of the vehicle;

generating a virtual viewpoint image where an environment of the vehicle is viewed from a virtual viewpoint, based on the acquired captured image;

compositing a notification image that provides notification of existence of the obstacle at the detected position with the virtual viewpoint image and compositing a vehicle image with the virtual viewpoint image; and

compositing the vehicle image with the virtual viewpoint image composited with the notification image, while overlapping the vehicle image and the notification image with each other, to form a composited vehicle image, where (i) at least a part of an overlapped part in the composited vehicle image between the vehicle and the obstacle is transparent, (ii) the vehicle and the notification overlap each other in the composited vehicle image, and (iii) in the overlapped part of the composited vehicle image, the vehicle image is transparent in order to not hide the notification image so that the notification image is viewed through the transparent vehicle image,

wherein acquiring the information regarding the detected position of the obstacle that exists in the environment of the vehicle includes (i) segmenting a detection range in the environment into a plurality of regions defined by intersecting longitudinal areas and latitudinal areas, (ii) determining the longitudinal area where the obstacle exists in the detection range, (iii) determining the latitudinal area where the obstacle exists in the detection range, and (iv) identifying a target region where the obstacle exists in the detection range based on where the determined longitudinal area and the determined latitudinal area intersect.