Radar system using a machine-learned model for stationary object detection
This document describes techniques and systems related to a radar system using a machine-learned model for stationary object detection. The radar system includes a processor that can receive radar data as time-series frames associated with electromagnetic (EM) energy. The processor uses the radar data to generate a range-time map of the EM energy that is input to a machine-learned model. The machine-learned model can receive as inputs extracted features corresponding to the stationary objects from the range-time map for multiple range bins at each of the time-series frames. In this way, the described radar system and techniques can accurately detect stationary objects of various sizes and extract critical features corresponding to the stationary objects.
1. A system comprising:
a radar system of a vehicle, the radar system including one or more processors configured to detect stationary objects by:
receiving radar data, the radar data comprising multiple time-series frames associated with electromagnetic (EM) energy reflected by one or more objects in a roadway and received by an antenna of the radar system, each time-series frame of the radar data including multiple range bins;
generating, using the time-series frames of the radar data, a Doppler beam vector of the EM energy;
generating, using the Doppler beam vector and a super-resolution operation, a range-azimuth map of the EM energy for each time-series frame of multiple time-series frames;
generating, using the range-azimuth map, an interpolated range-azimuth map of the EM energy by:
determining, for each range bin of multiple range bins within the range-azimuth map, a range-bin time, the range-bin time being the time-series frame at which the vehicle reaches the range bin;
selecting, for a respective range-bin time, bracketing data captures of the radar data that have a time-series frame immediately before and immediately after the respective range-bin time;
determining a range position of the vehicle at the bracketing data captures; and
shifting range values of the bracketing data captures in a range dimension so that a location of the vehicle and a stationary object among the one or more objects match those in an interpolated time-series frame;
generating, using the interpolated range-azimuth map, a range-time map of the EM energy, the range-time map representing a potential detection of the stationary object among the one or more objects as a straight, 45-degree line; and
detecting, using a machine-learned model, the stationary objects among the one or more objects, the machine-learned model being configured to receive as input extracted features corresponding to the one or more objects from the range time-map for multiple range bins at each of the time-series frames, the extracted features including at least two of range, target angle, velocity, geometrical features, or intensity-based features of the one or more objects.
2. The system of claim 1 , wherein the super-resolution operation can include at least one of a Fourier transform or an iterative adaptive approach to produce azimuth-angle data at each range bin.
3. The system of claim 1 , wherein the one or more processors are configured to shift the bracketing data captures in the range dimension by:
using a weighted average of the range values of the bracketing data captures, the weighted average based on relative differences between the time-series frames of the bracketing data captures and the interpolated time frame; or
selecting a filtered range value from the range values of the bracketing data captures.
4. The system of claim 1 , wherein the one or more processors are further configured to process the EM energy received by the antenna by:
down-sampling the range-azimuth map in range by taking a filtered intensity at each azimuth point over a first number of consecutive range bins to effectively compress the range-azimuth map.
5. The system of claim 4 , wherein the one or more processors are further configured to process the EM energy received by the antenna by:
enlarging a range-bin size of the range-azimuth map by a factor of the first number to effectively reduce a quantity of interpolated time frames by the factor of the first number.
6. The system of claim 1 , wherein the time-series frames include information associated with the stationary objects in multiple dimensions including at least three of a range dimension, a Doppler dimension, an elevation-angle dimension, or an azimuth-angle dimension.
7. The system of claim 1 , wherein the extracted features include at least two of range, target angle, velocity, geometrical features, or intensity-based features of the stationary objects.
8. The system of claim 1 , wherein:
the machine-learned model comprises a long short-term memory (LSTM) network with multiple layers.
9. A method comprising:
receiving radar data, the radar data comprising multiple time-series frames associated with electromagnetic (EM) energy reflected by one or more objects in a roadway and received by an antenna of a radar system of a vehicle, each time-series frame of the radar data including multiple range bins;
generating, using the time-series frames of the radar data, a Doppler beam vector of the EM energy;
generating, using the Doppler beam vector and a super-resolution operation, a range-azimuth map of the EM energy for each time-series of the multiple time-series frames;
generating, using the range-azimuth map, an interpolated range-azimuth map of the EM energy by:
determining, for each range bin of multiple range bin within the range-azimuth map, a range-bin time, the range-bin time being the time-series frame at which the vehicle reaches the range bin;
selecting, for a respective range-bin time, bracketing data captures of the radar data that have a time-series frame immediately before and immediately after the respective range-bin time;
determining a range position of the vehicle at the bracketing data captures; and
shifting range values of the bracketing data captures in a range dimension so that a location of the vehicle and a stationary object among the one or more objects match those in an interpolated time-series frame;
generating, using the interpolated range-azimuth map, a range-time map of the EM energy, the range-time map representing a potential detection of the stationary object among the one or more objects as a straight, 45-degree line; and
detecting, using a machine-learned model, stationary objects in a roadway among the one or more objects, the machine-learned model being configured to receive as input extracted features corresponding to the one or more objects from the range-time map for multiple range bins at each of the time-series frames, the extracted features including at least two of range, target angle, velocity, geometrical features, or intensity-based features of the one or more objects.
10. The method of claim 9 , wherein shifting the bracketing data captures in the range dimension comprises:
using a weighted average of the range values of the bracketing data captures, the weighted average based on relative differences between the time-series frames of the bracketing data captures and the interpolated time frame; or
selecting a filtered range value from the range values of the bracketing data captures.
11. The method of claim 9 , wherein processing the EM energy received by the antenna of the radar system further comprises:
down-sampling the range-azimuth map in range by taking a filtered intensity at each azimuth point over a first number of consecutive range bins to effectively compress the range-azimuth map.
12. The method of claim 9 , wherein processing the EM energy received by the antenna of the radar system further comprises:
enlarging a range-bin size of the range-azimuth map by a factor of the first number to effectively reduce quantity of interpolated time frames by the factor of the first number.
13. The method of claim 9 , wherein the time-series frames include information associated with the stationary objects in multiple dimensions including at least three of a range dimension, a Doppler dimension, an elevation-angle dimension, or an azimuth-angle dimension.
14. A non-transitory computer-readable storage medium comprising computer-executable instructions that, when executed, cause a processor of a radar system of a vehicle to:
receive radar data, the radar data comprising time-series frames associated with electromagnetic (EM) energy reflected by one or more objects in a roadway and received by an antenna of the radar system, each time-series frame of the radar data including multiple range bins;
generate, using the time-series frames of the radar data, a Doppler beam vector of the EM energy;
generate, using the Doppler beam vector and a super-resolution operation, a range-azimuth map of the EM energy for each time-series frame of the multiple time-series frames;
generate, using the range-azimuth map, an interpolated range-azimuth map of the EM energy by:
determining, for each range bin of multiple range bins within the range-azimuth map, a range-bin time, the range-bin time being the time-series frame at which the vehicle reaches the range bin;
selecting, for a respective range-bin time, bracketing data captures of the radar data that have a time-series frame immediately before the immediately after the respective range-bin time;
determining a range position of the vehicle at the bracketing data captures; and
shifting range values of the bracketing data captures in a range dimension so that a location of the vehicle and a stationary object among the one or more objects match those in an interpolated time-series frame;
generate, using the interpolated range-azimuth map, a range-time map of the EM energy, the range-time map representing a potential detection of the stationary object among the one or more objects as a straight, 45-degree line; and
detect, using a machine-learned model, stationary objects among the one or more objects, the machine-learned model being configured to receive as input extracted features corresponding to the one or more objects from the range time-map for multiple range bins at each of the time-series frames, the extracted features including at least two of range, target angle, velocity, geometrical features, or intensity-based features of the one or more objects.