Information processing device, vehicle, roadside unit, and information processing method
An information processing device includes an image interface, a sensor interface, and a controller. The image interface acquires a captured image from an imaging device. The sensor interface acquires a distance distribution from a distance measurement sensor that has a detection range that overlaps at least partially with the imaging range of the imaging device. The distance measurement sensor measures a distance distribution by receiving a reflected wave of a projected wave at an object. The distance distribution includes distance values to a reflection point in each of a plurality of directions. The controller detects a subject image from the captured image, and calculates the position of a subject, which corresponds to the subject image, in the width direction based on the captured image and the distance distribution. The width direction is parallel to the horizontal direction and perpendicular to the optical axis direction of the imaging device.
1 . An information processing device comprising:
an image interface configured to acquire a captured image from an imaging device;
a sensor interface configured to acquire a distance distribution from a distance measurement sensor which has a detection range overlapping at least partially with an imaging range of the imaging device and which is configured to measure the distance distribution including distance values to a reflection point in each of a plurality of directions by receiving a reflected wave of a projected wave at an object; and
a controller configured to detect a subject image from the captured image, calculate, on the basis of the captured image and the distance distribution, a position of a subject corresponding to the subject image in a width direction which is parallel to a horizontal direction and perpendicular to an optical axis direction of the imaging device, and prioritize use of the distance distribution to calculate a distance to the subject by changing a relative priority of the distance distribution to be used to calculate the distance to the subject according to a luminance of the captured image.
2 . The information processing device according to claim 1 , wherein
the captured image includes at least one of a visible light image, a near-infrared image, or a far-infrared image, and the distance distribution includes a detection result from at least one of radar, sonar, or LiDAR.
3 . The information processing device according to claim 1 ,
wherein the controller is configured to
identify, on the basis of the distance distribution, an area in the captured image where the subject image is highly likely to exist, and
detect the subject image from the captured image after executing at least one of expanding a luminance gain of the captured image or relaxing a threshold value to be used to detect the subject image in the identified area.
4 . The information processing device according to claim 3 , wherein
the controller is configured to identify, as the area where the subject image is highly likely to exist, an area in the captured image that corresponds to a direction group corresponding to a distance value group from among the plurality of direction-specific distance values in the distance distribution, the distance value group containing distance values that differ by a predetermined value or less and having directions that are close to each other.
5 . The information processing device according to claim 1 , wherein
the controller is configured to
create an object-specific area frame to surround the subject image through subject image detection in the captured image, and
calculate the distance of the subject on the basis of corresponding directions in the distance distribution within the object-specific area frame.
6 . The information processing device according to claim 5 , wherein the controller is configured to calculate the position in the width direction on the basis of the object-specific area frame and the distance of the subject.
7 . A vehicle comprising:
an imaging device configured to generate a captured image;
a distance measurement sensor which has a detection range overlapping at least partially with an imaging range of the imaging device and which is configured to generate a distance distribution including distance values to a reflection point in each of a plurality of directions by receiving a reflected wave of a projected wave at an object; and
a controller configured to detect a subject image from the captured image, calculate, on the basis of the captured image and the distance distribution, a position of a subject corresponding to the subject image in a width direction which is parallel to a horizontal direction and perpendicular to an optical axis direction of the imaging device, and prioritize use of the distance distribution to calculate a distance to the subject by changing a relative priority of the distance distribution to be used to calculate the distance to the subject according to a luminance of the captured image.
8 . A roadside unit comprising:
an imaging device configured to generate a captured image;
a distance measurement sensor which has a detection range overlapping at least partially with an imaging range of the imaging device and which is configured to generate a distance distribution including distance values to a reflection point in each of a plurality of directions by receiving a reflected wave of a projected wave at an object; and
a controller configured to detect a subject image from the captured image, calculate, on the basis of the captured image and the distance distribution, a position of a subject corresponding to the subject image in a width direction which is parallel to a horizontal direction and perpendicular to an optical axis direction of the imaging device, and prioritize use of the distance distribution to calculate a distance to the subject by changing a relative priority of the distance distribution to be used to calculate the distance to the subject according to a luminance of the captured image.
9 . An information processing method comprising:
generating a captured image;
generating a distance distribution including distance values to a reflection point in each of a plurality of directions by receiving a reflected wave of a projected wave at an object in a detection range that overlaps at least partially with a range of the captured image;
detecting a subject image from the captured image;
calculating, on the basis of the captured image and the distance distribution, a position of a subject corresponding to the subject image in a width direction which is parallel to a horizontal direction and perpendicular to an optical axis direction of an imaging device configured to generate the captured image; and
calculating a distance to the subject by prioritizing use of the distance distribution by changing a relative priority of the distance distribution to be used to calculate the distance to the subject according to a luminance of the captured image.
10 . The information processing device according to claim 1 , wherein
the controller is configured to change the relative priority by changing the weighting of the distance distribution to be used to calculate the distance to a subject according to the luminance of the captured image.
11 . The vehicle according to claim 7 , wherein
the controller is configured to change the relative priority by changing the weighting of the distance distribution to be used to calculate the distance to a subject according to the luminance of the captured image.
12 . The roadside unit according to claim 8 , wherein
the controller is configured to change the relative priority by changing the weighting of the distance distribution to be used to calculate the distance to a subject according to the luminance of the captured image.
13 . The information processing method according to claim 9 , wherein
the calculating the distance changes the relative priority by changing the weighting of the distance distribution to be used to calculate the distance to a subject according to the luminance of the captured image.
14 . The information processing device according to claim 1 , wherein
the controller is configured to execute at least one of expanding a luminance gain or relaxing a threshold value only in an area where a subject is highly likely to exist on the basis of a distance distribution.
15 . The roadside unit according to claim 8 , wherein
the controller is configured to execute at least one of expanding a luminance gain or relaxing a threshold value only in an area where a subject is highly likely to exist on the basis of a distance distribution.
16 . The information processing method according to claim 9 , further comprising:
executing at least one of expanding a luminance gain or relaxing a threshold value only in an area where a subject is highly likely to exist on the basis of a distance distribution.