Self-supervised learning
In some embodiments, the method may include determining, using a machine learning model, an estimated velocity corresponding to an object based at least on measured RADAR data, where the measured RADAR data may correspond to RADAR detections associated with the object. In some embodiments, the method may further include determining expected RADAR data corresponding to the object based at least on the estimated velocity. Some embodiments may additionally include updating one or more parameters of the machine learning model based on the difference between the measured RADAR data and the expected RADAR data.
1 . A method comprising:
obtaining, using a machine learning (ML) model, an output of the ML model that includes an estimated velocity corresponding to an object as a whole, the estimated velocity being based at least on providing measured RADAR data to the ML model as an input to the ML model, the measured RADAR data corresponding to a plurality of RADAR detections associated with different portions of the object;
reconstructing, to generate expected RADAR data as a reconstruction of the measured RADAR data and formatted according to a format of the measured RADAR data, RADAR data corresponding to the different portions of the object based at least on one or more reverse calculations performed with respect to the estimated velocity; and
updating one or more parameters of the ML model based at least on a difference between the measured RADAR data and the expected RADAR data, wherein the ML model is used by a machine to determine a detected velocity of a detected object detected using one or more RADAR sensors of the machine.
2 . The method of claim 1 , wherein the estimated velocity corresponding to the object includes one or more estimated velocities individually corresponding to one or more portions of the object.
3 . The method of claim 2 , wherein the one or more respective reverse calculations are performed with respect to the one or more estimated velocities.
4 . The method of claim 1 , wherein the estimated velocity is estimated prior to an ability to estimate a velocity corresponding to the object by identifying a plurality of bounding shapes corresponding to the object, individual bounding shapes of the plurality of bounding shapes associated with respective estimations of respective positions corresponding to the object.
5 . The method of claim 1 , wherein the measured RADAR data corresponds to a plurality of RADAR sensors.
6 . The method of claim 1 , wherein the ML model is updated to iteratively generate one or more additional estimated velocities until the difference between the measured RADAR data and the expected RADAR data reaches a particular threshold.
7 . A system comprising:
one or more processors to perform operations comprising:
obtaining RADAR data corresponding to an object, the RADAR data captured within a time range and corresponding to a plurality of RADAR sensors;
estimating, using a machine learning (ML) model, a velocity corresponding to the object based at least on the RADAR data corresponding to the plurality of RADAR sensors, wherein the ML model was trained based at least on reconstructing, to generate expected RADAR data, previously measured RADAR data from one or more previously estimated velocities previously estimated by the ML model as one or more previous outputs generated using the ML model based at least on the previously measured RADAR data being provided to the ML model as one or more inputs to the ML model, the expected RADAR data being of a same data type as the previously measured RADAR data; and
associating the velocity as estimated with the object detected using one or more object tracking techniques.
8 . The system of claim 7 , wherein the time range is within 50 milliseconds.
9 . The system of claim 7 , wherein the ML model was trained at least by:
determining an error based at least on a difference between the expected RADAR data and the previously measured RADAR data; and
updated one or more parameters of the M model based at least on the error.
10 . The system of claim 9 , wherein the ML model is updated to iteratively generate the one or more previously estimated velocities until the difference between the previously measured RADAR data and the expected RADAR data reaches a particular threshold.
11 . The system of claim 7 , wherein the expected RADAR data is generated based at least on one or more respective reverse calculations from the one or more previously estimated velocities.
12 . The system of claim 7 , wherein the velocity corresponding to the object, as estimated, includes one or more estimated velocities individually corresponding to one or more portions of the object.
13 . The system of claim 7 , wherein the system is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system for presenting at least one of augmented reality content, virtual reality content, or mixed reality content;
a system for hosting one or more real-time streaming applications;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system implementing one or more large language models (LLMs);
a system for generating synthetic data;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
14 . One or more processors comprising processing circuitry to perform one or more operations using a velocity estimate generated using a machine learning (ML) model, wherein the ML model is trained by:
determining, using the ML model, an estimated velocity corresponding to an object as a whole based at least on providing, as an input to the ML model, measured RADAR data corresponding to a plurality of RADAR detections associated with different portions of the object;
reconstructing the measured RADAR data according to a format of the measured RADAR data based at least on the estimated velocity to determine expected RADAR data corresponding to the different portions of the object; and
updating one or more parameters of the ML model based at least on a difference between the measured RADAR data and the expected RADAR data, wherein the ML model is used by a machine to determine a detected velocity of a detected object detected using one or more RADAR sensors of the machine.
15 . The one or more processors of claim 14 , wherein the estimated velocity corresponding to the object includes one or more estimated velocities individually corresponding to one or more portions of the object.
16 . The one or more processors of claim 14 , wherein the one or more operations further comprise:
generating a control command based at least on the estimated velocity corresponding to the object, the control command directing one or more systems associated with the one or more processors to perform the one or more operations.
17 . The one or more processors of claim 14 , wherein the estimated velocity corresponding to the object is estimated prior to an ability to estimate a velocity corresponding to the object by identifying a plurality of bounding shapes corresponding to the object, individual bounding shapes of the plurality of bounding shapes associated with respective estimations of respective positions corresponding to the object.
18 . The one or more processors of claim 14 , wherein the estimated velocity corresponding to the object includes one or more estimated velocities individually corresponding to one or more portions of the object; and
the determining the expected RADAR data is based at least on one or more respective reverse calculations from the one or more estimated velocities.
19 . The one or more processors of claim 14 , wherein the ML model is updated to iteratively generate one or more additional estimated velocities until the difference between the measured RADAR data and the expected RADAR data reaches a particular threshold.
20 . The one or more processors of claim 14 included in a system, wherein the system is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system for presenting at least one of augmented reality content, virtual reality content, or mixed reality content;
a system for hosting one or more real-time streaming applications;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system implementing one or more large language models (LLMs);
a system for generating synthetic data;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.