Information processing device, information processing method, and program
The present disclosure relates to an information processing device, an information processing method, and a program for improving object recognition accuracy. A feature amount obtained by a point cloud that is a dense detection point of LiDAR or the like and a feature amount of a sparse detection point of a millimeter wave radar or the like within a predetermined distance are grouped and integrated with reference to the position of the feature amount of the dense detection point, the point cloud is restored using an integrated feature amount of the dense detection point, and object recognition processing such as semantic segmentation is performed. The present disclosure can be applied to a mobile body.
1. An information processing device comprising:
a first acquisition unit configured to acquire a first feature amount that is a feature amount of a detection point detected by a first sensor;
a second acquisition unit configured to acquire a second feature amount that is a feature amount of a detection point detected by a second sensor; and
an integrated feature amount calculation unit configured to calculate an integrated feature amount obtained by integrating the first feature amount and the second feature amount on a basis of a relative distance between the respective detection points of the first feature amount and the second feature amount, wherein the integrated feature amount calculation unit
further includes a corresponding measurement information calculation unit configured to calculate a relative position between the detection point of the first feature amount and the detection point of the second feature amount as corresponding measurement information,
calculates the relative distance between the respective detection points of the first feature amount and the second feature amount on a basis of information of the relative position, and
calculates the integrated feature amount obtained by integrating the first feature amount and the second feature amount on a basis of the calculated distance.
2. The information processing device according to claim 1 , wherein
the corresponding measurement information calculation unit calculates the relative position between the detection point of the first feature amount and the detection point of the second feature amount as the corresponding measurement information, and generates an integration table in which
the first feature amount and the second feature amount are associated with each other, and the integrated feature amount calculation unit calculates the relative distance between the respective detection points of the first feature amount and the second feature amount on a basis of the integration table, and calculates the integrated feature amount obtained by integrating the first feature amount and the second feature amount on a basis of the calculated distance.
3. The information processing device according to claim 2 , wherein
the integrated feature amount calculation unit further includes a selection unit configured to calculate the relative distance between the respective detection points of the first feature amount and the second feature amount on a basis of the integration table, and select the second feature amount within a predetermined distance as a feature amount to be grouped, for each of the first feature amounts, from among the second feature amounts, and calculates the integrated feature amount obtained by integrating the second feature amount selected as the feature amount to be grouped as the second feature amount within the predetermined distance, for each of the first feature amounts, by the selection unit, and the first feature amount.
4. The information processing device according to claim 3 , wherein
the integrated feature amount calculation unit further includes an integration unit configured to calculate the integrated feature amount obtained by integrating the second feature amount selected as the feature amount to be grouped as the second feature amount within the predetermined distance, for each of the first feature amounts, by the selection unit, and the first feature amount, and outputs the integrated feature amount calculated by the integration unit.
5. The information processing device according to claim 1 , comprising:
a plurality of the integrated feature amount calculation units.
6. The information processing device according to claim 1 , wherein
the first sensor and the second sensor have different resolutions.
7. The information processing device according to claim 6 , wherein
the first feature amount and the second feature amount have different sparseness and denseness.
8. The information processing device according to claim 6 , wherein
the resolution of the first sensor is higher than the resolution of the second sensor.
9. The information processing device according to claim 8 , wherein the first feature amount is denser than the second feature amount.
10. The information processing device according to claim 6 , wherein
the first sensor is a LiDAR, and the second sensor is a millimeter wave radar.
11. The information processing device according to claim 1 , wherein
the distance is a spatial distance.
12. The information processing device according to claim 1 , wherein
the distance is a temporal distance.
13. The information processing device according to claim 1 , wherein
the distance is a distance obtained by combining both a spatial distance and a temporal distance.
14. The information processing device according to claim 1 , further comprising:
a local feature amount calculation unit configured to sample the integrated feature amount such that each spatial distance becomes longer, group the sampled integrated feature amount and another integrated feature amount within a spatially predetermined distance, integrate the sampled integrated feature amount and the grouped integrated feature amount, and calculate a local feature amount.
15. The information processing device according to claim 14 , further comprising:
a global feature amount calculation unit configured to restore a feature amount corresponding to the integrated feature amount that has been lost due to the integration of the sampled integrated feature amount and the grouped integrated feature amount by the local feature amount calculation unit, on a basis of the local feature amount, and calculate the restored feature amount as a global feature amount.
16. The information processing device according to claim 15 , wherein
the local feature amount calculation unit and the global feature amount calculation unit form a PointNet++structure.
17. An information processing method of an information processing device including a first acquisition unit, a second acquisition unit, and an integrated feature amount calculation unit, the information processing method comprising the steps of:
by first acquisition unit, acquiring a first feature amount that is a feature amount of a detection point detected by a first sensor;
by the second acquisition unit, acquiring a second feature amount that is a feature amount of a detection point detected by a second sensor; and
by the integrated feature amount calculation unit, calculating an integrated feature amount obtained by integrating the first feature amount and the second feature amount on a basis of a relative distance between the respective detection points of the first feature amount and the second feature amount, calculating a relative position between the detection point of the first feature amount and the detection point of the second feature amount as corresponding measurement information, calculating the relative distance between the respective detection points of the first feature amount and the second feature amount on a basis of information of the relative position, and calculating the integrated feature amount obtained by integrating the first feature amount and the second feature amount on a basis of the calculated distance.
18. A non-transitory computer-readable storage medium storing a program that, when processed by a processor of a computer, causes the computer to function as:
a first acquisition unit configured to acquire a first feature amount that is a feature amount of a detection point detected by a first sensor;
a second acquisition unit configured to acquire a second feature amount that is a feature amount of a detection point detected by a second sensor; and
an integrated feature amount calculation unit configured to calculate an integrated feature amount obtained by integrating the first feature amount and the second feature amount on a basis of a relative distance between the respective detection points of the first feature amount and the second feature amount, wherein the integrated feature amount calculation unit
further includes a corresponding measurement information calculation unit configured to calculate a relative position between the detection point of the first feature amount and the detection point of the second feature amount as corresponding measurement information,
calculates the relative distance between the respective detection points of the first feature amount and the second feature amount on a basis of information of the relative position, and
calculates the integrated feature amount obtained by integrating the first feature amount and the second feature amount on a basis of the calculated distance.