Obstacle detection apparatus, obstacle detection method, and obstacle detection program
In an obstacle detection apparatus, the captured image of a vicinity of a vehicle from an imaging apparatus is acquired. A three-dimensional estimation image showing a three-dimensional position of a feature point in the captured image is generated, and a three-dimensional position of an object is estimated. An attribute image in which an object in the captured image is classified into one or more classes that include at least a road-surface class is generated. The three-dimensional estimation image and the attribute image are fused, the feature points and the classes are associated, and road-surface points associated with the road-surface class are extracted. Based on the road-surface points, a road-surface height in the vicinity of the vehicle is estimated. Based on the estimated road-surface height, the three-dimensional position of the object is corrected. Based on the three-dimensional position of the object, an obstacle in the vicinity of the vehicle is detected.
1. An obstacle detection apparatus that detects an obstacle in a vicinity of a vehicle based on a captured image of the vicinity of the vehicle captured by an imaging apparatus, the obstacle detection apparatus comprising one or more processors configured to implement:
an information acquiring unit that acquires the captured image of the vicinity of the vehicle;
a three-dimensional estimating unit that generates a three-dimensional estimation image that shows a three-dimensional position of a feature point in the captured image and estimates a three-dimensional position of an object in the vicinity of the vehicle based on the three-dimensional estimation image;
an image classifying unit that generates an attribute image in which an object in the captured image is classified into one or more classes that include at least a road-surface class;
a road-surface point extracting unit that fuses the three-dimensional estimation image and the attribute image, associates the feature points and the classes, and extracts road-surface points that are the feature points associated with the road-surface class;
a road-surface height estimating unit that estimates a road-surface height in the vicinity of the vehicle based on the road-surface points, the estimating of the road-surface height including:
determining reliability of a frequency distribution of a road-surface-point height, and;
estimating, when it is determined that the frequency distribution is reliable, the road-surface height based on the frequency distribution of the road-surface-point height;
a correcting unit that corrects the three-dimensional position of the object in the vicinity of the vehicle based on the estimated road-surface height; and
an obstacle detecting unit that detects an obstacle in the vicinity of the vehicle based on the three-dimensional position of the object in the vicinity of the vehicle.
2. The obstacle detection apparatus according to claim 1 , wherein:
the road-surface height is estimated in response to a number of extracted road-surface points being equal to or greater than a predetermined first threshold.
3. The obstacle detection apparatus according to claim 1 , wherein:
the correcting unit corrects mounting information of the imaging apparatus based on changes over time in the road-surface height that is estimated by the road-surface height estimating unit.
4. An obstacle detection apparatus that detects an obstacle in a vicinity of a vehicle based on a captured image of the vicinity of the vehicle captured by an imaging apparatus, the obstacle detection apparatus comprising one or more processors configured to implement:
an information acquiring unit that acquires the captured image of the vicinity of the vehicle;
a three-dimensional estimating unit that generates a three-dimensional estimation image that shows a three-dimensional position of a feature point in the captured image and estimates a three-dimensional position of an object in the vicinity of the vehicle based on the three-dimensional estimation image;
an image classifying unit that generates an attribute image in which an object in the captured image is classified into one or more classes that include at least a road-surface class;
a road-surface point extracting unit that fuses the three-dimensional estimation image and the attribute image, associates the feature points and the classes, and extracts road-surface points that are the feature points associated with the road-surface class;
a road-surface height estimating unit that estimates a road-surface height in the vicinity of the vehicle based on the road-surface points that are continuously associated with the road-surface class for a predetermined period;
a correcting unit that corrects the three-dimensional position of the object in the vicinity of the vehicle based on the estimated road-surface height; and
an obstacle detecting unit that detects an obstacle in the vicinity of the vehicle based on the three-dimensional position of the object in the vicinity of the vehicle.
5. The obstacle detection apparatus according to claim 4 , wherein:
the road-surface height estimating unit estimates the road-surface height in response to a number of extracted road-surface points being equal to or greater than a predetermined first threshold.
6. An obstacle detection apparatus that detects an obstacle in a vicinity of a vehicle based on a captured image of the vicinity of the vehicle captured by an imaging apparatus, the obstacle detection apparatus comprising one or more processors configured to implement:
an information acquiring unit that acquires the captured image of the vicinity of the vehicle;
a three-dimensional estimating unit that generates a three-dimensional estimation image that shows a three-dimensional position of a feature point in the captured image and estimates a three-dimensional position of an object in the vicinity of the vehicle based on the three-dimensional estimation image;
an image classifying unit that generates an attribute image in which an object in the captured image is classified into one or more classes that include at least a road-surface class;
a road-surface point extracting unit that fuses the three-dimensional estimation image and the attribute image, associates the feature points and the classes, and extracts road-surface points that are the feature points associated with the road-surface class;
a road-surface height estimating unit that estimates a road-surface height in the vicinity of the vehicle based on the road-surface points and that provides an estimation result of the road-surface height together with information that estimation reliability is low in response to the number of road-surface points that are used for estimation of the road-surface height being less than a predetermined second threshold;
a correcting unit that corrects the three-dimensional position of the object in the vicinity of the vehicle based on the estimated road-surface height; and
an obstacle detecting unit that detects an obstacle in the vicinity of the vehicle based on the three-dimensional position of the object in the vicinity of the vehicle.
7. An obstacle detection apparatus that detects an obstacle in a vicinity of a vehicle based on a captured image of the vicinity of the vehicle captured by an imaging apparatus, the obstacle detection apparatus comprising one or more processors configured to implement:
an information acquiring unit that acquires the captured image of the vicinity of the vehicle;
a three-dimensional estimating unit that generates a three-dimensional estimation image that shows a three-dimensional position of a feature point in the captured image and estimates a three-dimensional position of an object in the vicinity of the vehicle based on the three-dimensional estimation image;
an image classifying unit that generates an attribute image in which an object in the captured image is classified into one or more classes that include at least a road-surface class;
a road-surface point extracting unit that fuses the three-dimensional estimation image and the attribute image, associates the feature points and the classes, and extracts road-surface points that are the feature points associated with the road-surface class;
a road-surface height estimating unit that estimates a road-surface height in the vicinity of the vehicle based on the road-surface points and that provides an estimation result of the road-surface height together with information that estimation reliability is high in response to the number of road-surface points that are used for estimation of the road-surface height being equal to or greater than a predetermined second threshold;
a correcting unit that corrects the three-dimensional position of the object in the vicinity of the vehicle based on the estimated road-surface height; and
an obstacle detecting unit that detects an obstacle in the vicinity of the vehicle based on the three-dimensional position of the object in the vicinity of the vehicle.
8. The obstacle detection apparatus according to claim 7 , wherein:
the correcting unit corrects mounting information of the imaging apparatus based on changes over time in the road-surface height that is estimated by the road-surface height estimating unit.
9. An obstacle detection method that is applicable to an obstacle detection apparatus that detects an obstacle in a vicinity of a vehicle based on a captured image of the vicinity of the vehicle captured by an imaging apparatus, the obstacle detection method comprising:
an information acquiring step of acquiring a captured image of the vicinity of the vehicle;
a three-dimensional estimating step of generating a three-dimensional estimation image that shows a three-dimensional position of a feature point in the captured image and estimating a three-dimensional position of an object in the vicinity of the vehicle based on the three-dimensional estimation image;
an image classifying step of generating an attribute image in which an object in the captured image is classified into one or more classes that include at least a road-surface class;
a road-surface point extracting step of fusing the three-dimensional estimation image and the attribute image, associating the feature points and the classes, and extracting road-surface points that are the feature points associated with the road-surface class;
a road-surface height estimating step of estimating a road-surface height in the vicinity of the vehicle based on the road-surface points, the road-surface height estimating step comprising:
determining reliability of a frequency distribution of a road-surface-point height, and
estimating, when determined that the frequency distribution is reliable, the road-surface height based on the frequency distribution of the road-surface-point height;
a correcting step of correcting the three-dimensional position of the object in the vicinity of the vehicle based on the estimated road-surface height; and
an obstacle detecting step of detecting an obstacle in the vicinity of the vehicle based on the three-dimensional position of the object in the vicinity of the vehicle.
10. A non-transitory computer-readable storage medium storing therein an obstacle detection program that includes processes that are performed by a processor to detect an obstacle in a vicinity of a vehicle based on a captured image of the vicinity of the vehicle captured by an imaging apparatus, the processes including:
an information acquiring step of acquiring a captured image of the vicinity of the vehicle;
a three-dimensional estimating step of generating a three-dimensional estimation image that shows a three-dimensional position of a feature point in the captured image and estimating a three-dimensional position of an object in the vicinity of the vehicle based on the three-dimensional estimation image;
an image classifying step of generating an attribute image in which an object in the captured image is classified into one or more classes that include at least a road-surface class;
a road-surface point extracting step of fusing the three-dimensional estimation image and the attribute image, associating the feature points and the classes, and extracting road-surface points that are the feature points associated with the road-surface class;
a road-surface height estimating step of estimating a road-surface height in the vicinity of the vehicle based on the road-surface points, the road-surface height estimating step comprising
determining reliability of a frequency distribution of a road-surface-point height, and
estimating, when determined that the frequency distribution is reliable, the road-surface height based on the frequency distribution of the road-surface-point height;
a correcting step of correcting the three-dimensional position of the object in the vicinity of the vehicle based on the estimated road-surface height; and
an obstacle detecting step of detecting an obstacle in the vicinity of the vehicle based on the three-dimensional position of the object in the vicinity of the vehicle.