Systems and methods for producing amodal cuboids
Systems and methods for operating an autonomous vehicle. The methods comprising: obtaining, by a computing device, loose-fit cuboids overlaid on 3D graphs so as to each encompass LiDAR data points associated with a given object; defining, by the computing device, an amodal cuboid based on the loose-fit cuboids; using, by the computing device, the amodal cuboid to train a machine learning algorithm to detect objects of a given class using sensor data generated by sensors of the autonomous vehicle or another vehicle; and causing, by the computing device, operations of the autonomous vehicle to be controlled using the machine learning algorithm.
1. A method for operating an autonomous vehicle, comprising:
obtaining, by a computing device, loose-fit cuboids overlaid on 3D graphs so as to each encompass LiDAR data points associated with a given object;
defining, by the computing device, an amodal cuboid based on the loose-fit cuboids;
using, by the computing device, the amodal cuboid to train a machine learning algorithm to detect objects of a given class using sensor data generated by sensors of the autonomous vehicle or another vehicle; and
causing, by the computing device, operations of the autonomous vehicle to be controlled using the machine learning algorithm.
2. The method according to claim 1 , further comprising:
determining, by the computing device, whether the given object is less than a first distance from a LiDAR sensor;
wherein the amodal cuboid is used to train the machine learning algorithm when a determination is made that the given object is less than the first distance from the LiDAR sensor, and at least one another cuboid is used when a determination is made that the given object is greater than the first distance from the LiDAR sensor.
3. The method according to claim 2 , wherein the at least one another cuboid comprises the loose-fit cuboids or a weighted cuboid generated based on the loose-fit cuboids and the amodal cuboid.
4. The method according to claim 1 , wherein the defining comprises determining coordinates for a center of the amodal cuboid based on at least one of a tight-fit cuboid or a loose-fit cuboid of the loose-fit cuboids.
5. The method according to claim 4 , wherein the center of the amodal cuboid is determined by: (i) aligning a corner of the amodal cuboid with a closest visible corner of the tight-fit cuboid; and (ii) aligning a corner of the amodal cuboid such that a boundary precision between the amodal cuboid and the loose-fit cuboid is maximized.
6. The method according to claim 1 , wherein the defining comprises identifying LiDAR data points inside one of the loose-fit cuboids.
7. The method according to claim 6 , wherein the defining comprising removing data points from the LiDAR data points that are associated with ground.
8. The method according to claim 6 , wherein the defining comprises defining a tight-fit cuboid that tightly fits the LiDAR data points.
9. The method according to claim 8 , wherein the defining comprises accumulating LiDAR data points in the loose-fit cuboids.
10. The method according to claim 9 , wherein the defining comprising defining an accumulated-fit cuboid that tightly fits the accumulated LiDAR data points of the loose-fit cuboids.
11. The method according to claim 10 , wherein the defining comprises determining an amodal extent using the accumulated-fit cuboid and prior amodal extents for an object.
12. The method according to claim 11 , wherein the amodal cuboid has dimensions specified by the amodal extent.
13. The method according to claim 11 , wherein the amodal extent is set equal to a maximum prior size of a prior amodal extent when a size of the accumulated-fit cuboid is greater than the maximum prior size.
14. The method according to claim 11 , wherein the amodal extent is set equal to a minimum size of a prior amodal extent when a size of the accumulated-fit cuboid is less than the minimum prior size.
15. The method according to claim 11 , wherein the amodal extent is set equal to an extent of the accumulated-fit cuboid when a size of the amodal extent falls between a maximum prior size and a minimum prior size of the prior amodal extents.
16. A system, comprising:
a processor;
a non-transitory computer-readable storage medium comprising programming instructions that are configured to cause the processor to implement a method for operating a vehicle, wherein the programming instructions comprise instructions to:
obtain loose-fit cuboids overlaid on 3 D graphs so as to each encompass LiDAR data points associated with a given object;
define an amodal cuboid based on the loose-fit cuboids;
use the amodal cuboid to train a machine learning algorithm to detect objects of a given class using sensor data generated by sensors of the autonomous vehicle or another vehicle; and
cause operations of the autonomous vehicle to be controlled using the machine learning algorithm.
17. The system according to claim 16 , wherein:
the programming instructions further comprise instructions to determine whether the given object is less than a first distance from a LiDAR sensor; and
the amodal cuboid is used to train the machine learning algorithm when a determination is made that the given object is less than the first distance from the LiDAR sensor, and at least one another cuboid is used when a determination is made that the given object is greater than the first distance from the LiDAR sensor.
18. The system according to claim 17 , wherein the at least one another cuboid comprises the loose-fit cuboids or a weighted cuboid generated based on the loose-fit cuboids and the amodal cuboid.
19. The system according to claim 16 , wherein the amodal cuboid is defined by determining coordinates for a center of the amodal cuboid based on at least one of a tight-fit cuboid or a loose-fit cuboid of the loose-fit cuboids.
20. The system according to claim 19 , wherein the center of the amodal cuboid is determined by: (i) aligning a corner of the amodal cuboid with a closest visible corner of the tight-fit cuboid;
and (ii) aligning a corner of the amodal cuboid such that a boundary precision between the amodal cuboid and the loose-fit cuboid is maximized.
21. The system according to claim 16 , wherein the amodal cuboid is defined by identifying LiDAR data points inside one of the loose-fit cuboids.
22. The system according to claim 21 , wherein the amodal cuboid is defined by removing data points from the LiDAR data points that are associated with ground.
23. The system according to claim 21 , wherein the amodal cuboid is defined by defining a tight-fit cuboid that tightly fits the LiDAR data points.
24. The system according to claim 23 , wherein the amodal cuboid is defined by accumulating LiDAR data points in the loose-fit cuboids.
25. The system according to claim 24 , wherein the amodal cuboid is defined by defining an accumulated-fit cuboid that tightly fits the accumulated LiDAR data points of the loose-fit cuboids.
26. The system according to claim 25 , wherein the amodal cuboid is defined by determining an amodal extent using the accumulated-fit cuboid and prior amodal extents for an object.
27. The sysetm according to claim 25 , wherein the amodal cuboid has dimensions specified by the amodal extent.
28. The system according to claim 25 , wherein the amodal extent is set equal to a maximum prior size of a prior amodal extent when a size of the accumulated-fit cuboid is greater than the maximum prior size.
29. The system according to claim 25 , wherein the amodal extent is set equal to a minimum size of a prior amodal extent when a size of the accumulated-fit cuboid is less than the minimum prior size.
30. The system according to claim 25 , wherein the amodal extent is set equal to an extent of the accumulated-fit cuboid when a size of the amodal extent falls between a maximum prior size and a minimum prior size of the prior amodal extents.