Super-resolution radar for autonomous vehicles
Examples disclosed herein relate to an autonomous driving system in an vehicle. The autonomous driving system includes a radar system configured to detect a target in a path and a surrounding environment of the vehicle and produce radar data with a first resolution that is gathered over a continuous field of view on the detected target. The system includes a super-resolution network configured to receive the radar data with the first resolution and produce radar data with a second resolution different from the first resolution using first neural networks. The system also includes a target identification module configured to receive the radar data with the second resolution and to identify the detected target from the radar data with the second resolution using second neural networks. Other examples disclosed herein include a method of operating the radar system in the autonomous driving system of the vehicle.
1. A method for a radar system, comprising:
receiving radar data at a first time having a first resolution;
storing the radar data and associated timestamp in memory;
receiving lidar data at the first time having a second resolution different from the first resolution, wherein the first resolution is coarser than the second resolution;
storing the lidar data in a sparse lidar point cloud arrangement data format in memory;
transforming the lidar data into densified data in a denser data format subdivide a field of view of the radar system into regularly spaced voxels of infinite height transforming the densified data onto three-dimensional data based on the subdivided field of view;
forming a training dataset from the radar data and the densified data;
processing the training dataset by applying a random rotation to the training dataset to form an augmented training dataset;
transforming the radar data by mapping the radar data to the densified data using the training dataset; and
processing features of the radar data to output data having features substantially corresponding to features of the densified data.
2. The method of claim 1 wherein a value of each regularly spaced voxel corresponds to a difference in height between high and low points in voxel bounds.
3. The method of claim 2 , further comprising:
training a machine learning network with the training dataset.
4. The method of claim 3 , wherein processing features of coarse-resolution radar data comprising providing radar data as input to the machine learning network.
5. The method of claim 4 , wherein the machine learning network is a neural network type architecture.
6. The method of claim 3 , further comprising:
after training on the training data set, the machine learning network receiving radar data at a coarse resolution and mapping to less coarse resolution.
7. The method of claim 1 , further comprising:
providing the radar having the second resolution to a target detection module.
8. A radar system, comprising:
a radar module;
a sensor module;
a receiver adapted to receive radar data at a first time having a first resolution;
a data processing module adapted to receive sensor data at the first time having a second resolution different from the first resolution and further adapted to:
form a training dataset from the radar data and the sensor data, wherein the sensor data is densified data;
map the radar data having the first resolution to radar data having the second resolution using the training dataset, wherein the first resolution is a coarse resolution compared to the second resolution;
densify the sensor data in the training dataset, wherein the radar data has a dense arrangement and the sensor data has a sparse point cloud arrangement;
map the densified data onto a three-dimensional perspective representation based on a field of view, the field of view subdivided into regularly spaced voxels of infinite height; and
apply a random rotation to the training dataset to augment the training dataset; and
process features of coarse-resolution radar data to output data having features substantially corresponding to features of fine-resolution sensor data.
9. The radar system of claim 8 , further comprising:
an occupancy detection module to identify targets in a field of view.
10. The radar system of claim, 8 , wherein the sensor data forms a sparse point cloud of data.
11. The radar system of claim 8 , further comprising:
an antenna controller adapted to subdivide a field of view of a vehicle; and
wherein the resolution network associates each subdivision with a voxel of space.
12. The radar system of claim 11 , wherein the sensor module is a lidar module.
13. The radar system of claim 11 , the resolution network is a machine learning network.
14. The radar system of claim 13 , wherein the machine learning network is a neural network type network.
15. The radar system of claim 13 , wherein the target identification module comprises:
a Doppler module to detect a velocity of a target using the mapped radar data;
a range module to detect a range of the target using the mapped radar data; and
a decision module to determine a next action based on the range and velocity of the target.