Path perception diversity and redundancy in autonomous machine applications
In various examples, a path perception ensemble is used to produce a more accurate and reliable understanding of a driving surface and/or a path there through. For example, an analysis of a plurality of path perception inputs provides testability and reliability for accurate and redundant lane mapping and/or path planning in real-time or near real-time. By incorporating a plurality of separate path perception computations, a means of metricizing path perception correctness, quality, and reliability is provided by analyzing whether and how much the individual path perception signals agree or disagree. By implementing this approach—where individual path perception inputs fail in almost independent ways—a system failure is less statistically likely. In addition, with diversity and redundancy in path perception, comfortable lane keeping on high curvature roads, under severe road conditions, and/or at complex intersections, as well as autonomous negotiation of turns at intersections, may be enabled.
1 . A method comprising:
determining, based at least on sensor data obtained using one or more sensors of a machine, a first line representation corresponding to a first line feature in an environment of the machine;
determining, based at least on map data, a second line representation that corresponds to a second line feature in the environment of the machine;
determining, based at least on a first classification associated with the first line representation and a second classification associated with the second line representation, that the first line feature and the second line feature correspond to a same line feature;
generating, using the first line representation and the second line representation, a third line representation corresponding to the same line feature; and
causing the machine to perform one or more operations based at least on the third line representation.
2 . The method of claim 1 , wherein the generating the third line representation comprises generating, using the first line representation and the second line representation, the third line representation as being located at least partially between the first line representation and the second line representation.
3 . The method of claim 1 , wherein the generating the third line representation comprises:
applying first data representative of the first line representation and second data representative of the second line representation to one or more neural networks; and
generating, using the one or more neural networks and based at least on the first data and the second data, third data representative of the third line representation.
4 . The method of claim 1 , wherein the generating the third line representation comprises:
determining one or more differences between the first line representation and the second line representation; and
determining the third line representation based at least on the one or more differences.
5 . The method of claim 1 , further comprising:
determining that the first line representation is within a threshold distance to the second line representation,
wherein the determining that the first line feature and the second line feature correspond to the same line feature is further based at least on the first line representation being within the threshold distance to the second line representation.
6 . The method of claim 1 , wherein the determining the first line representation comprises:
applying the sensor data to one or more neural networks; and
generating, using the one or more neural networks and based at least on the sensor data, output data representative of the first line representation.
7 . The method of claim 1 , wherein the determining the first line representation comprises:
determining, based at least on the sensor data, a path associated with an object; and
determining the first line representation based at least on the path.
8 . The method of claim 1 , further comprising:
converting the first line representation to a coordinate system; and
converting the second line representation to the coordinate system,
wherein the generating the third line representation is based at least on the first line representation and the second line representation as converted within the coordinate system.
9 . The method of claim 1 , further comprising:
determining a confidence associated with the third line representation,
wherein the causing the machine to perform the one or more operations is further based at least on the confidence.
10 . A system comprising:
one or more processors to:
determine, based at least on first sensor data obtained using a machine, a first representation associated with a driving surface;
determine, based at least on second sensor data obtained using the machine, a second representation associated with the driving surface;
determine, based at least on the first representation being within a threshold distance to the second representation, a correspondence between the first representation and the second representation;
determine, using the first representation and the second representation and based at least on the correspondence, a third representation associated with the driving surface; and
cause the machine to perform one or more operations based at least on the third representation.
11 . The system of claim 10 , wherein:
the first representation comprises one or more first points associated with the driving surface;
the second representation comprises one or more second points associated with the driving surface; and
the third representation comprises one or more third points associated with the driving surface, at least a third point of the one or more third points being different than the one or more first points and the one or more second points.
12 . The system of claim 10 , wherein the determination of the third representation comprises generating, using the first representation and the second representation and based at least on the correspondence, the third representation as being located at least partially between the first representation and the second representation.
13 . The system of claim 10 , wherein the determination of the third representation comprises:
determining one or more differences between the first representation and the second representation; and
determining the third representation based at least on the one or more differences.
14 . The system of claim 10 , wherein the one or more processors are further to:
determine that a first classification associated with the first representation includes a same classification as a second classification associated with the second representation,
wherein the determination of the correspondence between the first representation and the second representation is further based at last on the first classification including the same classification as the second classification.
15 . The system of claim 10 , wherein:
the first representation comprises one or more of a first lane graph, a first lane representation, a first lane rail, a first lane edge, first lane dimensions, a first line, one or more first points, or one or more first trajectories associated with one or more first objects; and
the second representation comprises one or more of a second lane graph, a second lane representation, a second lane rail, a second lane edge, second lane dimensions, a second line, one or more second points, or one or more second trajectories associated with one or more second objects.
16 . The system of claim 10 , wherein:
the first sensor data is obtained using a first type of sensor associated with the machine; and
the second sensor data is obtained using a second type of sensor associated with the machine, the second type of sensor being different than the first type of sensor.
17 . The system of claim 10 , wherein the one or more processors are further to:
convert the first representation to a coordinate system; and
convert the second representation to the coordinate system,
wherein the third representation is further determined based at least on the first representation and the second representation as converted within the coordinate system.
18 . The system of claim 10 , wherein the system is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for the autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing real-time streaming;
a system for generating or presenting virtual reality (VR) content;
a system for generating or presenting augmented reality (AR) content;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
19 . One or more processors comprising:
processing circuitry to:
determine, based at least on first sensor data obtained using a machine, a first representation associated with a driving surface;
determine, based at least on second sensor data obtained using the machine, a second representation associated with the driving surface;
determine, based at least on a first classification associated with the first representation including a same classification as a second classification associated with the second representation, that the first representation corresponds to the second representation;
determine, based at least on the first representation corresponding to the second representation, a third representation associated with the driving surface; and
cause the machine to perform one or more operations based at least on the third representation.
20 . The one or more processors of claim 19 , wherein the one or more processors are comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for the autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing real-time streaming;
a system for generating or presenting virtual reality (VR) content;
a system for generating or presenting augmented reality (AR) content;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.