Object detection for autonomous systems and applications
In various examples, one or more object detectors may regress bounding polygons for detected objects in systems (e.g., autonomous or semi-autonomous driving systems and applications) that provide object awareness, object identification, object avoidance, and/or object localization. The object detector may determine regression data representing a regressed polygon associated with a given shape of a detected object represented by classification data determined from a scene. The object detector may determine regression data for different regressed angles between different pairs of successive vertices of the regressed polygon and regressed lengths of vectors from a regressed geometric center of the regressed polygon to vertices of the regressed polygon. The object detector may generate, based at least in part on the regression data, a bounding shape for a detected object in the scene. In some embodiments, the object detector may be trained by deforming a regressed polygon to match a ground truth polygon.
1 . A processor comprising:
one or more circuits to:
determine, using one or more neural networks and based at least on sensor data corresponding to a scene, regression data representing, for at least one regressed polygon corresponding to one or more shapes of one or more detected objects, at least a regressed angle between vertices of the at least one regressed polygon;
generate, based at least in part on the regression data, one or more bounding shapes of the one or more detected objects in the scene; and
perform one or more operations based at least on the one or more bounding shapes.
2 . The processor of claim 1 , wherein the regression data further represents different regressed angles between different pairs of successive vertices of the at least one regressed polygon.
3 . The processor of claim 1 , wherein the one or more circuits are further to determine the regressed angle as an angular offset from a regressed origin of the at least one regressed polygon, the regressed origin being a geometric center of the at least one regressed polygon located at a regressed offset from a corresponding cell of the regression data representing the at least one regressed polygon.
4 . The processor of claim 1 , wherein the one or more circuits are further to generate the one or more bounding shapes based at least on selecting a cell of classification data representing or more classifications in the scene, the cell of classification data having a threshold classification value and generating a bounding shape of the one or more bounding shapes based at least on a depth-wise vector associated with a corresponding cell of the regression data.
5 . The processor of claim 1 , wherein the regression data further represents at least a regressed length of a vector of one or more vectors from a regressed geometric center of the at least one regressed polygon to a corresponding one of the vertices.
6 . The processor of claim 1 , wherein the one or more neural networks are trained based at least on deforming regressed polygons during training to have a same number of vertices as candidate ground truth polygons.
7 . The processor of claim 1 , wherein the one or more neural networks are trained based at least on deforming regressed polygons during training to have a same angular offset between successive vertices as candidate ground truth polygons.
8 . The processor of claim 1 , wherein the one or more neural networks are trained based at least on deforming regressed polygons during training to have a same vector length from origin to each vertex of one or more of the vertices as candidate ground truth polygons.
9 . The processor of claim 1 , wherein the one or more neural networks are trained based at least on matching regressed polygons during training with candidate ground truth polygons using a Hungarian algorithm.
10 . The processor of claim 1 , wherein the one or more detected objects represented by one or more classifications in the scene correspond to one or more of a feature, a dynamic object, a static object, a crosswalk, a lane marking, a road boundary marking, a road marking, a sign, a traffic light, a vehicle, a pedestrian, a cyclist, or a robot.
11 . The processor of claim 1 , wherein the processor is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system for presenting at least one of augmented reality content, virtual reality content, or mixed reality content;
a system for hosting one or more real-time streaming applications;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system implementing one or more large language models (LLMs);
a system for generating synthetic data;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
12 . A method comprising:
determining, using one or more neural networks and based at least on sensor data corresponding to a scene, classification data representing one or more classifications in the scene;
determining, using the one or more neural networks, regression data representing, for at least one regressed polygon corresponding to one or more shapes of one or more detected objects indicated by the one or more classifications in the scene, different regressed angles between different pairs of successive vertices of the at least one regressed polygon;
generating, based at least in part on the regression data, one or more bounding shapes of the one or more detected objects in the scene; and
performing one or more operations based at least on the one or more bounding shapes.
13 . The method of claim 12 , further comprising determining the different regressed angles as angular offsets from a regressed origin of the at least one regressed polygon, the regressed origin being a geometric center of the at least one regressed polygon located at a regressed offset from a corresponding cell of the regression data representing the at least one regressed polygon.
14 . The method of claim 12 , further comprising generating the one or more bounding shapes based at least on selecting a cell of the classification data having a threshold classification value and generating a bounding shape of the one or more bounding shapes based at least on a depth-wise vector associated with a corresponding cell of the regression data.
15 . The method of claim 12 , wherein the one or more neural networks are trained based at least on deforming regressed polygons during training to have a same number of vertices as candidate ground truth polygons.
16 . The method of claim 12 , wherein the one or more neural networks are trained based at least on deforming regressed polygons during training to have: a same vector length from origin to each vertex of one or more vertices as candidate ground truth polygons, and a same angular offset between vertices as the candidate ground truth polygons.
17 . The method of claim 12 , wherein the one or more neural networks are trained based at least on matching regressed polygons with candidate ground truth polygons using a Hungarian algorithm.
18 . The method of claim 12 , wherein the one or more detected objects in the scene correspond to one or more of a feature, a dynamic object, a static object, a crosswalk, a lane marking, a road boundary marking, a road marking, a sign, a traffic light, a vehicle, a pedestrian, a cyclist, or a robot.
19 . The method of claim 12 , wherein the method is executed using at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system for presenting at least one of augmented reality content, virtual reality content, or mixed reality content;
a system for hosting one or more real-time streaming applications;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system implementing one or more large language models (LLMs);
a system for generating synthetic data;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
20 . A system comprising:
one or more processing units to cause performance of one or more operations associated with a machine based at least on one or more regressed polygons corresponding to one or more detected objects in a scene, the one or more regressed polygons determined based at least on one or more regressed angles between one or more vertices of the one or more regressed polygons as computed using one or more machine learning models.