Information processing apparatus, information processing method, program, mobile-object control apparatus, and mobile object
The present technology relates to an information processing apparatus, an information processing method, a program, a mobile-object control apparatus, and a mobile object that make it possible to improve the accuracy in recognizing a target object. An information processing apparatus includes an image processor that generates an estimated-location image on the basis of a sensor image that indicates, in a first coordinate system, a sensing result of a sensor of which a sensing range at least partially overlaps a sensing range of an image sensor, the estimated-location image indicating an estimated location of a target object in a second coordinate system identical to a coordinate system of a captured image obtained by the image sensor; and an object recognition section that performs processing of recognizing the target object on the basis of the captured image and the estimated-location image. The present technology is applicable to, for example, a system used to recognize a target object around a vehicle.
1. An information processing apparatus, comprising:
at least one processor configured to:
generate, using a model obtained by machine learning, a first transformed sensor data based at least in part on first sensor data generated by a first sensor, wherein a sensing range of the first sensor at least partially overlaps a sensing range of a second sensor;
generate, using a second model obtained by machine learning, a second transformed sensor data based at least in part on second sensor data generated by the second sensor; and
recognize a target object based at least in part on the first transformed sensor data and the second transformed sensor data.
2. The information processing apparatus according to claim 1 , wherein:
the target object is recognized based at least in part on the first transformed sensor data, the second transformed sensor data, and third sensor data generated by a third sensor; and
the model obtained by machine learning is caused to perform learning using training data that includes input data and ground truth data, the input data including the first sensor data, the ground truth data indicating a location of the target object in the third sensor data.
3. The information processing apparatus according to claim 2 , wherein
the ground truth data is a binary image that indicates a region, in the third sensor data, in which there exists the target object.
4. The information processing apparatus according to claim 2 , wherein
the model obtained by machine learning is a model using a deep neural network.
5. The information processing apparatus according to claim 4 , wherein
the model obtained by machine learning includes:
a feature-amount extraction section that extracts a feature amount of the first sensor data to generate a feature map indicating a distribution of the feature amount in a first coordinate system,
a geometric transformation section that transforms the feature map in the first coordinate system into the feature map in a second coordinate system, wherein the second coordinate system is identical to a coordinate system of the third sensor data, and
a deconvolution section that deconvolves the feature map in the second coordinate system to generate the first transformed sensor data.
6. The information processing apparatus according to claim 2 , wherein
the target object is recognized using an object recognition model obtained by machine learning.
7. The information processing apparatus according to claim 6 , wherein
the object recognition model is caused to perform learning using training data that includes input data and ground truth data, the input data including the third sensor data and the first transformed sensor data, the ground truth data indicating a location of the target object in the third sensor data.
8. The information processing apparatus according to claim 7 , wherein
the object recognition model is a model using a deep neural network.
9. The information processing apparatus according to claim 8 , wherein
the object recognition model includes:
a first convolutional neural network that extracts feature amounts of the first transformed sensor data and the third sensor data, and
a second convolutional neural network that recognizes the target object based at least in part on the feature amounts of the first transformed sensor data and the third sensor data.
10. The information processing apparatus according to claim 6 , wherein
the first sensor and the second sensor perform sensing with respect to surroundings of a mobile object, and
the object recognition model recognizes the target object in the surroundings of the mobile object.
11. The information processing apparatus according to claim 1 , wherein
the first sensor includes a radar, and
the first sensor data indicates a location of an object off which a transmission signal from the radar is reflected.
12. The information processing apparatus according to claim 11 , wherein
a first coordinate system of the first sensor is defined by an axis that represents an angle to an optical-axis direction of the radar, and by an axis that represents a distance to the object.
13. The information processing apparatus according to claim 11 , wherein
the radar has a resolving power in a height direction, and
the processor is configured to generate the first transformed sensor data on a basis of a plurality of sensor images corresponding to different heights.
14. The information processing apparatus according to claim 1 , wherein
the first sensor includes light detection and ranging (LiDAR), and
the first sensor data is group-of-points data obtained by the LiDAR.
15. An information processing method, comprising:
using at least one processor to perform:
generating, using a model obtained by machine learning, a first transformed sensor data, based at least in part on first sensor data generated by a first sensor, wherein a sensing range of the first sensor at least partially overlaps a sensing range of a second sensor;
generating a second transformed sensor data, using a second model obtained by machine learning, based at least in part on second sensor data generated by the second sensor; and
recognizing a target object based at least in part on the first transformed sensor data and second transformed sensor data.
16. At least one non-transitory computer readable storage medium, storing a program for causing a computer to perform a process, the process comprising:
generating, using a model obtained by machine learning, a first transformed sensor data, based at least in part on first sensor data generated by a first sensor, wherein a sensing range of the first sensor at least partially overlaps a sensing range of a second sensor;
generating a second transformed sensor data, using a second model obtained by machine learning, based at least in part on second sensor data generated by the second sensor; and
recognizing a target object based at least in part on the first transformed sensor data and second transformed sensor data.
17. The information processing apparatus of claim 1 , wherein the target object is recognized based at least in part on the first transformed sensor data, the second transformed sensor data, and third sensor data generated by a third sensor.
18. A system comprising:
a first sensor configured to generate first sensor data;
a second sensor configured to generate second sensor data, wherein a sensing range of the first sensor at least partially overlaps with a sensing range of the second sensor; and
at least one processor configured to:
generate, using a machine learning model, first transformed sensor data based at least in part on the first sensor data;
generate, using a second machine learning model, a second transformed sensor data based at least in part on the second sensor data; and
recognize a target object based at least in part on the first transformed sensor data and the second transformed sensor data.