IP Library › Granted Patent US 9,873,379
Granted Patent B2
US 9,873,379 · App. 14/643,072 · Granted Jan 23, 2018

Composite image generation apparatus and composite image generation program

Inventor: Masanari Takaki (Chiryu, JP)
Assignee: DENSO CORPORATION
B60R1/00G06T3/4038H04N5/23238B60R2300/303B60R2300/607
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,873,379
App. No.
14/643,072
Granted
Jan 23, 2018
Kind
B2
Abstract

In a composite image generation apparatus that is mounted in an own vehicle, each of captured images, which have been captured by a plurality of imaging units, are acquired. A disturbance level for each of the plurality of captured images that have been acquired is determined. The disturbance level indicates: whether or not a disturbance is present in the captured image; or an extent of the disturbance. In an overlapping area in which imaging areas of the plurality of captured images overlap, one or more captured images is selected from the plurality of captured images so that the area occupied by a captured image having a higher disturbance level among the plurality captured images is smaller. A composite image is generated based on the plurality of captured images. In the overlapping area, the composite image is generated by using the captured image selected based on the disturbance level.

Claims (77)

1. A composite image generation apparatus that is mounted in an own vehicle and generates a composite image which is a combination of captured images acquired by a plurality of imaging units, the composite image generation apparatus comprising:

a processor configured to

acquire each of captured images that have been captured by the plurality of imaging units;

determine a disturbance level for each of the plurality of captured images that have been acquired, the disturbance level indicating an extent of a disturbance;

select, in an overlapping area in which imaging areas of the plurality of captured images overlap, one or more captured images from the plurality of captured images so that the area occupied by a captured image having a higher disturbance level among the plurality captured images is smaller; and

generate a composite image based on the plurality of captured images, and generate the composite image by using the captured image selected based on the disturbance level in the overlapping area, wherein

the processor is configured to deduct a number of points from an allotted score corresponding to each of the plurality of captured images in response to the disturbance level.

2. The composite image generation apparatus according to claim 1 , wherein

the processor is configured to

determine the type of disturbance as at least a part of the disturbance level based on a brightness value of each pixel composing each of the captured images; and

select one or more captured images from the plurality of captured images based on the disturbance level.

3. The composite image generation apparatus according to claim 2 , wherein

the processor is configured to select one or more captured images that has the lowest disturbance level.

4. The composite image generation apparatus according to claim 3 , wherein

the processor is configured to:

determine the disturbance level expressed by numeric values; and

set an occupancy ratio of each of the captured images in the overlapping area based on a ratio of the disturbance level expressed by numeric values of each of the captured images.

5. The composite image generation apparatus according to claim 4 , wherein

the processor is configured to generate, as the composite image, a bird's-eye view image that is supposed to be viewed from directly above the own vehicle.

6. The composite image generation apparatus according to claim 5 , wherein

the processor is further configured to extract a road surface paint from the composite image.

7. The composite image generation apparatus according to claim 6 , wherein

the processor is further configured to

calculate, based on the disturbance level, a proportion of captured images in which a disturbance has occurred, among the plurality of captured images; and

prohibit an operation of the processor that extracts road surface paint when determined that the proportion is a threshold or higher.

8. The composite image generation apparatus according to claim 1 , wherein

the processor is configured to select one or more captured images that has the lowest disturbance level.

9. The composite image generation apparatus according to claim 1 , wherein

the processor is configured to

determine the disturbance level expressed by numeric values; and

set an occupancy ratio of each of the captured images in the overlapping area based on a ratio of the disturbance level expressed by numeric values of each of the captured images.

10. The composite image generation apparatus according to claim 1 , wherein

the processor is configured to generate, as the composite image, a bird's-eye view image that is supposed to be viewed from directly above the own vehicle.

11. A composite image generation apparatus that is mounted in an own vehicle and generates a composite image which is a combination of captured images acquired by a plurality of imaging units, the composite image generation apparatus comprising:

a processor configured to

acquire each of captured images that have been captured by the plurality of imaging units;

determine a disturbance level for each of the plurality of captured images that have been acquired, the disturbance level indicating an extent of a disturbance;

select, in an overlapping area in which imaging areas of the plurality of captured images overlap, one or more captured images from the plurality of captured images so that the area occupied by a captured image having a higher disturbance level among the plurality captured images is smaller; and

generate a composite image based on the plurality of captured images, and generate the composite image by using the captured image selected based on the disturbance level in the overlapping area, wherein

the processor is further configured to extract a road surface paint from the composite image, and

the processor is further configured to

calculate, based on the disturbance level, a proportion of captured images in which a disturbance has occurred, among the plurality of captured images; and

prohibit an operation of the processor that extracts road surface paint when determined that the proportion is a threshold or higher.

12. An image display system comprising:

a plurality of imaging units that is mounted in an own vehicle and acquires captured images;

a composite image generation apparatus that is mounted in the own vehicle and generates a composite image which is a combination of captured images acquired by the plurality of imaging units; and

a display that is mounted in the own vehicle and displays the composite image generated by the composite image generation apparatus,

the composite image generation apparatus comprising

a processor configured to

acquire each of captured images that have been captured by the plurality of imaging units;

determine a disturbance level for each of the plurality of captured images that have been acquired, the disturbance level indicating an extent of a disturbance;

select, in an overlapping area in which imaging areas of the plurality of captured images overlap, one or more captured images from the plurality of captured images so that the area occupied by a captured image having a higher disturbance level among the plurality captured images is smaller; and

generate a composite image based on the plurality of captured images, and generate the composite image by using the captured image selected based on the disturbance level in the overlapping area, wherein

the processor is configured to deduct a number of points from an allotted score corresponding to each of the plurality of captured images in response to the disturbance level.

13. The image display system according to claim 12 , wherein the plurality of imaging units are a plurality of cameras mounted to the own vehicle.

14. The image display system according to claim 12 , wherein the processor of the composite image generating apparatus determines the disturbance level for each of the plurality of captured images by determining a brightness value of each pixel that composes each of the plurality of captured images.

15. The image display system according to claim 14 , wherein the processor of the composite image generating apparatus determines a type of disturbance based on the brightness value of each pixel that composes each of the plurality of captured images.

16. The image display system according to claim 14 , wherein the processor determines whether the brightness value is saturated.

17. The image display system according to claim 2 , wherein the processor determines whether the brightness value is saturated.

18. A composite image generation apparatus that is mounted in an own vehicle and generates a composite image which is a combination of captured images acquired by a plurality of imaging units, the composite image generation apparatus comprising:

a processor configured to

acquire each of captured images that have been captured by the plurality of imaging units;

determine a disturbance level for each of the plurality of captured images that have been acquired, the disturbance level indicating an extent of a disturbance;

select, in an overlapping area in which imaging areas of the plurality of captured images overlap, one or more captured images from the plurality of captured images so that the area occupied by a captured image having a higher disturbance level among the plurality captured images is smaller; and

generate a composite image based on the plurality of captured images, and generate the composite image by using the captured image selected based on the disturbance level in the overlapping area, wherein

the processor is further configured to

determine the type of disturbance as at least a part of the disturbance level based on a brightness value of each pixel composing each of the captured images; and

select one or more captured images from the plurality of captured images based on the disturbance level, and

the processor is further configured to select one or more captured images that has the lowest disturbance level,

the processor is further configured to

determine the disturbance level expressed by numeric values; and

set an occupancy ratio of each of the captured images in the overlapping area based on a ratio of the disturbance level expressed by numeric values of each of the captured images,

the processor is further configured to generate, as the composite image, a bird's-eye view image that is supposed to be viewed from directly above the own vehicle,

the processor is further configured to extract a road surface paint from the composite image, and

the processor is further configured to

calculate, based on the disturbance level, a proportion of captured images in which a disturbance has occurred, among the plurality of captured images; and

prohibit an operation of the processor that extracts road surface paint when determined that the proportion is a threshold or higher.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 15, 2015
From: TAKAKI, MASANARI
To: DENSO CORPORATION
Reel/Frame 035440/0386 →
Priority Claims (1)
JP 2014-048892 · Mar 12, 2014 · national
Continuity (1)
Related Publication 20150258936A1 · Sep 17, 2015