Systems and methods for estimating cuboid headings based on heading estimations generated using different cuboid defining techniques
Disclosed herein are systems, methods, and computer program products for operating a robotic system. For example, the method includes: obtaining a first cuboid generated based on an image, a second cuboid generated based on a lidar dataset and/or a third cuboid generated by a heuristic algorithm using the lidar dataset; using a machine learning model to generate a heading for an object in proximity to the robotic system based on the first cuboid, second cuboid and/or third cuboid; generating a bounding box geometry and a bounding box location based on the second cuboid or third cuboid; and generating a fourth cuboid using the bounding box geometry, the bounding box location, and the heading generated using the machine learning model.
1. A method for operating a robotic system, comprising:
obtaining, by a computing device, at least two of a first cuboid generated based on an image, a second cuboid generated based on a lidar dataset, and a third cuboid generated by a heuristic algorithm using the lidar dataset;
using, by the computing device, a machine learning model to generate a heading for an object in proximity to the robotic system based on at least two of the first cuboid, second cuboid and third cuboid;
generating, by the computing device, a bounding box geometry and a bounding box location based on the second cuboid or third cuboid; and
generating, by the computing device, a fourth cuboid using the bounding box geometry, the bounding box location, and the heading generated using the machine learning model; and
using, by the computing device, the fourth cuboid to cause movement of the robotic system.
2. The method according to claim 1 , further comprising generating a plurality of features for input into a machine learning algorithm based on at least two of the first cuboid, the second cuboid, the third cuboid and lane geometries.
3. The method according to claim 2 , wherein the plurality of features comprise a first heading of the first cuboid, a second heading of the second cuboid, and a third heading of the third cuboid.
4. The method according to claim 3 , wherein the plurality of features further comprise a variance for the first heading, a variance for the second heading, and a variance for the third heading.
5. The method according to claim 4 , wherein the plurality of features further comprise a value indicating an availability of a heading and a heading variance associated with the first cuboid, a value indicating an availability of a heading and a heading variance associated with the second cuboid, and a value indicating an availability of a heading and a heading variance associated with the third cuboid.
6. The method according to claim 2 , wherein the plurality of features comprise a distance from the robotic system to the first cuboid, a distance from the robotic system to the second cuboid, and a distance from the robotic system to the third cuboid.
7. The method according to claim 2 , wherein the plurality of features comprise a number of points in the second cuboid or third cuboid.
8. The method according to claim 2 , wherein the plurality of features comprise at least one of a course of the object, a plurality of lane directions, a value indicating whether the object is in a driving lane, a distance from the object to a driving lane, an area of the object based on a geometry of the third cuboid, a long side length of the object based on the geometry of the third cuboid, a short side length of the object based on the geometry of the third cuboid, and a maximum height of the object from a ground surface.
9. The method according to claim 1 , wherein the bounding box geometry and the bounding box location are generated by: selecting the second cuboid when available for the object; selecting the third cuboid when the second cuboid is not available for the object; and using a geometry of the second or third cuboid which was selected to define a size and location for a bounding box.
10. A system, comprising:
a processor;
a non-transitory computer-readable storage medium comprising programming instructions that are configured to cause the processor to implement a method for operating a robotic system, wherein the programming instructions comprise instructions to:
obtain at least two of a first cuboid generated based on an image, a second cuboid generated based on a lidar dataset, and a third cuboid generated by a heuristic algorithm using the lidar dataset;
using a machine learning model to generate a heading for an object in proximity to the robotic system based on at least two of the first cuboid, second cuboid and third cuboid;
generate a bounding box geometry and a bounding box location based on the second cuboid or third cuboid; and
generate a fourth cuboid using the bounding box geometry, the bounding box location, and the heading generated using the machine learning model; and
use the fourth cuboid to cause movement of the robotic system.
11. The system according to claim 10 , wherein the programming instructions further comprise instructions to generate a plurality of features for input into a machine learning algorithm based on at least two of the first cuboid, the second cuboid, the third cuboid and lane geometries.
12. The system according to claim 11 , wherein the plurality of features comprise a first heading of the first cuboid, a second heading of the second cuboid, and a third heading of the third cuboid.
13. The system according to claim 12 , wherein the plurality of features further comprise a variance for the first heading, a variance for the second heading, and a variance for the third heading.
14. The system according to claim 13 , wherein the plurality of features further comprise a value indicating an availability of a heading and a heading variance associated with the first cuboid, a value indicating an availability of a heading and a heading variance associated with the second cuboid, and a value indicating an availability of a heading and a heading variance associated with the third cuboid.
15. The system according to claim 11 , wherein the plurality of features comprise a distance from the robotic system to the first cuboid, a distance from the robotic system to the second cuboid, and a distance from the robotic system to the third cuboid.
16. The system according to claim 11 , wherein the plurality of features comprise a number of points in the second cuboid or third cuboid.
17. The system according to claim 11 , wherein the plurality of features comprise at least one of a course of the object, a plurality of lane directions, a value indicating whether the object is in a driving lane, a distance from the object to a driving lane, an area of the object based on a geometry of the third cuboid, a long side length of the object based on the geometry of the third cuboid, a short side length of the object based on the geometry of the third cuboid, and a maximum height of the object from a ground surface.
18. A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:
obtaining at least two of a first cuboid generated based on an image, a second cuboid generated based on a lidar dataset, and a third cuboid generated by a heuristic algorithm using the lidar dataset;
using a machine learning model to generate a heading for an object in proximity to the robotic system based on at least two of the first cuboid, second cuboid and third cuboid;
generating a bounding box geometry and a bounding box location based on the second cuboid or third cuboid;
generating a fourth cuboid using the bounding box geometry, the bounding box location, and the heading generated using the machine learning model; and
using the fourth cuboid to cause movement of the robotic system.