Use of camera information for radar beamforming

1 . An autonomous vehicle comprising one or more sensor systems to detect characteristics of an operating environment, the one or more sensor systems comprising at least a camera sensor system and a RADAR sensor system, the one or more sensor systems to:

receive, from the camera sensor system, camera information co-registered with a RADAR sensor system field of view of a scene having a target object;

segment the camera information into an image segment that includes the target object;

identify the target object from the camera information;

determine a target type and target location coordinates for the target object based on the image segment;

receive, from the RADAR sensor system, digital RADAR waveforms corresponding to a frame of reference of the RADAR, the frame of reference including the target object;

transform the target location coordinates for the target object to the frame of reference;

determine a radar cross section estimation for the target object based on a ratio of a structural dimension of the image segment to a wavelength of the RADAR system, wherein the structural dimension is based on the transformed coordinates;

select a kernel from a kernel repository based on the radar cross section estimation, target location coordinates, and the target type, wherein the selected kernel comprises a time series representation of the target type applicable to the scene;

perform data beamforming by convolving the digital RADAR waveforms utilizing the selected kernel to maximize output resolution corresponding to the camera coordinates and maximize the angular resolution based on the phase and magnitude of the RADAR waveforms; and

populate a point cloud with results from the RADAR signal processing.

2 . The autonomous vehicle of claim 1 , wherein the camera information for the scene comprises location information for at least one target and size information for the target object.

3 . The autonomous vehicle of claim 1 , wherein the camera information comprises azimuth and elevation information.

4 . The autonomous vehicle of claim 1 , wherein the RADAR signal processing comprises at least performing a convolution utilizing digital RADAR waveforms and the selected kernel.

5 . The autonomous vehicle of claim 1 , wherein the scene has multiple target objects, further comprising selecting a kernel for each of the multiple target objects.

6 . The autonomous vehicle of claim 1 , wherein the one or more sensor systems are further configured to populate a point cloud with results from the RADAR signal processing.

7 . A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, are configurable to cause the processors to:

receive camera information for a scene having a target object, wherein the camera information is co-registered with a RADAR sensor system field of view of the scene;

segment the camera information into an image segment that includes the target object;

identify the target object in the image segment from the camera information;

determine a target type and target location coordinates for the target object based on the image segment;

receive digital RADAR waveforms corresponding to a frame of reference of a RADAR system, the frame of reference including the target object;

transform the target location coordinates for the target object to the frame of reference;

determine a radar cross section estimation for the target object based on a ratio of a structural dimension of the image segment to a wavelength of the RADAR system, wherein the structural dimension is based on the transformed coordinates;

select a kernel from a kernel repository based on the radar cross section estimation, target location coordinates, and the target type, wherein the selected kernel comprises a time series representation of the target type applicable to the scene;

perform data beamforming by convolving the digital RADAR waveforms with the selected kernel to maximize output resolution corresponding to the target location coordinates and the angular resolution based on the phase and magnitude of the RADAR waveforms; and

populate a point cloud with results from the RADAR signal processing.

8 . The non-transitory computer-readable medium of claim 7 , wherein the camera information for the scene comprises location information for at least one target and size information for the target object.

9 . The non-transitory computer-readable medium of claim 7 , wherein the camera information comprises azimuth and elevation information.

10 . The non-transitory computer-readable medium of claim 7 , wherein the RADAR signal processing comprises at least performing a convolution utilizing digital RADAR waveforms and the selected kernel.

11 . The non-transitory computer-readable medium of claim 7 , wherein the scene has multiple target objects and the processor is further configured to select a kernel for each of the multiple target objects.

12 . The non-transitory computer-readable medium of claim 7 , wherein the target object comprises one of: a vehicle, a bicycle, and a pedestrian.

13 . A system comprising:

a memory system; and

one or more hardware processors coupled with the memory system, the one or more processors to:

receive camera information for a scene having a target object, wherein the camera information is co-registered with a RADAR sensor system field of view of the scene;

segment the camera information into an image segment that includes the target object;

identify the target object in the image segment from the camera information;

determine a target type and target location coordinates for the target object based on the image segment;

receive digital RADAR waveforms corresponding to a frame of reference of a RADAR system, the frame of reference including the target object;

transform the target location coordinates for the target object to the frame of reference;

determine a radar cross section estimation for the target object based on a ratio of a structural dimension of the image segment to a wavelength of the RADAR system, wherein the structural dimension is based on the transformed coordinates;

select a kernel from a kernel repository based on the radar cross section estimation, target location coordinates, and the target type, wherein the selected kernel comprises a time series representation of the target type applicable to the scene; and

perform data beamforming by convolving the digital RADAR waveforms with the selected kernel to maximize output resolution corresponding to the target location coordinates and the angular resolution based on the phase and magnitude of the RADAR waveforms.

14 . The system of claim 13 , wherein the camera information for the scene comprises location information for at least one target and size information for the target object.

15 . The system of claim 13 , wherein the camera information comprises azimuth and elevation information.

16 . The system of claim 13 , wherein the RADAR signal processing comprises at least performing a convolution utilizing digital RADAR waveforms and the selected kernel.

17 . The system of claim 13 , wherein the one or more hardware processors are further configured to populate a point cloud with results from the RADAR signal processing.