Sampling radar signals for automotive radar perception
In various examples, methods and systems are provided for sampling and transmitting the most useful information from a radar signal representing a scene while staying within the computational and storage confines of a standard automotive radar sensor and the bandwidth constraints of a standard communication link between a radar sensor and processing unit. Disclosed approaches may select a patch of frequency bins that correspond to radar signals based at least on proximities of the frequency bins to one or more frequency bins corresponding to at least one peak and/or detection point in the radar signals. Data representing samples corresponding to the patch of frequency bins may be transmitted to the processing unit and applied to one or more machine learning models in order to accurately classify, identify, and/or track objects.
1 . A method comprising:
determining local peaks in a representation of one or more radar signals, the representation being arranged as ordered intervals associated with frequencies of the one or more radar signals, and the local peaks collectively correspond to an object in an environment;
determining respective subsets of the ordered intervals for the local peaks based at least on selecting, for a local peak of the local peaks, a plurality of ordered intervals based at least on:
determining one or more quantities of the ordered intervals to include in the plurality of ordered intervals along one or more dimensions of the representation based at least on one or more spatial characteristics associated with the object as derived from sensor data corresponding to the object; and
evaluating proximities, within the representation, of the plurality of ordered intervals to an ordered interval that includes at least one point corresponding to the local peak;
generating samples corresponding to respective ordered intervals from the respective subsets of the ordered intervals determined for the local peaks; and
transmitting the samples to one or more processing units of a machine to cause the one or more processing units to use the samples to perform one or more perception operations corresponding to the object.
2 . The method of claim 1 , wherein respective sets of the samples correspond to respective local peaks of the local peaks, and the one or more perception operations include using the samples to perform object detection to detect the object.
3 . The method of claim 2 , wherein first sample values of a first set of the respective sets of the samples are normalized relative to a first local peak of the local peaks and second sample values of a second set of the respective sets of the samples are normalized relative to a second local peak of the local peaks.
4 . The method of claim 1 , wherein the generating includes downsampling the one or more radar signals corresponding to the respective subsets to determine one or more downsampled radar signals, wherein the samples correspond to the one or more downsampled radar signals.
5 . The method of claim 1 , wherein the generating the includes clipping the one or more radar signals corresponding to the respective subsets to determine one or more clipped radar signals, wherein the samples correspond to the one or more clipped radar signals.
6 . The method of claim 1 , wherein the generating includes compressing radar information corresponding to the respective subsets to determine compressed radar information, wherein the samples correspond to the compressed radar information.
7 . The method of claim 1 , wherein the plurality of ordered intervals further correspond to one or more of a Doppler, range, azimuth, or elevation dimension associated with the one or more radar signals.
8 . The method of claim 1 , wherein the plurality of ordered intervals include a first quantity of intervals along a first dimension of the ordered intervals and a second quantity of intervals different from the first quantity of intervals along a second dimension of the ordered intervals.
9 . The method of claim 1 , wherein a quantity of intervals included in the plurality of ordered intervals is based at least on a resolution in a dimension of at least one sensor used to generate the one or more radar signals.
10 . The method of claim 1 , wherein the transmitting is from one or more radar sensors used to capture the one or more radar signals to the one or more processing units, the samples include signal data corresponding to the one or more radar signals, and the one or more perception operations are performed using one or more machine learning models that process the signal data.
11 . A system comprising:
one or more radio detection and ranging (RADAR) sensors to generate radar signals; and
one or more processors to execute operations comprising:
determining a representation comprising ordered intervals associated with frequencies of the radar signals, the representation corresponding to the radar signals;
determining respective subsets of the ordered intervals for local peaks collectively correspond to an object in the representation based at least on selecting, for a local peak of the local peaks, a plurality of ordered intervals based at least on:
determining one or more quantities of the ordered intervals to include in the plurality of ordered intervals along one or more dimensions of the representation based at least on one or more spatial characteristics associated with the object as derived from sensor data corresponding to the object; and
evaluating proximities, within the representation, of the plurality of ordered intervals to an ordered interval that includes at least one point corresponding to the local peak;
computing samples corresponding to respective ordered intervals from respective subsets of the ordered intervals determined for the local peaks; and
transmitting the samples to one or more downstream components of a machine, the transmitting causing the machine to use the samples to perform one or more perception operations corresponding to the object.
12 . The system of claim 11 , wherein the ordered intervals correspond to one or more of a Doppler, range, azimuth, or elevation dimension associated with the radar signals.
13 . The system of claim 11 , wherein the radar signals comprise raw range-Doppler-angle fast Fourier Transform signals.
14 . The system of claim 11 , wherein the one or more downstream components include at least one processor.
15 . The system of claim 11 , wherein the computing the samples includes downsampling radar information corresponding to the ordered intervals to determine downsampled radar information, wherein the samples correspond to the downsampled radar information.
16 . At least one processor comprising:
one or more circuits to:
determine local peaks in a representation of one or more radar signals, the representation being arranged as ordered intervals associated with frequencies of the one or more radar signals, and the local peaks collectively correspond to an object in an environment;
determine respective subsets of the ordered intervals for the local peaks based at least on selecting, for a local peak of the local peaks, a plurality of ordered intervals based at least on;
determining one or more quantities of the ordered intervals to include in the plurality of ordered intervals along one or more dimensions of the representation based at least on one or more spatial characteristics associated with the object as derived from sensor data corresponding to the object; and
evaluating proximities, within the representation, of the plurality of ordered intervals to an ordered interval that includes at least one point corresponding to the local peak;
generate samples corresponding to respective ordered intervals from the respective subsets of the ordered intervals determined for the local peaks; and
transmit the samples to one or more processing units of a machine to cause the one or more processing units to use the samples to perform one or more perception operations corresponding to the object.
17 . The at least one processor of claim 16 , wherein the one or more perception operations include identifying the object.
18 . The at least one processor of claim 16 , wherein the one or more circuits are to identify each local peak of the local peaks based at least on comparing a magnitude of the local peak to a threshold value.
19 . The at least one processor of claim 16 , wherein the generating includes filtering the ordered intervals.
20 . The at least one processor of claim 16 , wherein the at least one processor is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.