Vehicles, systems, and methods for determining an entry of an occupancy map of a vicinity of a vehicle

1. A computer-implemented method comprising:

determining, using computer hardware components, an entry of an occupancy map of a vicinity of a vehicle by:

acquiring first sensor data of a first sensor in the vicinity of the vehicle;

acquiring second sensor data of a second sensor in the vicinity of the vehicle;

determining, based on the second sensor data, a segmented image using a segmentation neural network that determines an object class feature for each segment of the segmented image;

determining a first sensor patch of a pre-determined first sensor patch size from the first sensor data, the first sensor patch corresponding to a potential object in the vicinity of the vehicle;

based on the first sensor patch and the segmented image, determining a second sensor patch of a pre-determined second sensor patch size from the second sensor data, the second sensor patch corresponding to a location of the potential object;

extracting features of the first sensor patch;

extracting features of the second sensor patch including an object class feature for the second sensor patch; and

determining the entry of the occupancy map that corresponds to the potential object by fusing the features of the first sensor patch and the second sensor patch using a fusion artificial neural network.

2. The computer implemented method of claim 1 , wherein the first sensor comprises a radar sensor.

3. The computer implemented method of claim 2 , wherein the second sensor comprises a camera.

4. The computer implemented method of claim 1 , wherein at least one of:

the first sensor data comprises a plurality of measurements of the first sensor stacked over time; or

the second sensor data comprises a plurality of measurements of the second sensor stacked over time.

5. The computer implemented method of claim 1 , wherein the first sensor patch and the second sensor patch are multi-dimensional with identical size in at least one dimension.

6. The computer implemented method of claim 5 ,

wherein the first sensor patch and the second sensor patch have dimensions related to time, width, height, and features; and

wherein the first sensor patch and the second sensor patch have identical size in the dimensions related to width and height.

7. The computer implemented method of claim 1 , wherein the second sensor patch is determined so that an area in the vicinity of the vehicle covered by the second sensor patch is related to an area in the vicinity of the vehicle covered by the first sensor patch.

8. The computer implemented method of claim 7 , wherein the second sensor patch is determined based on mapping corner points of the first sensor patch to the second sensor data.

9. The computer implemented method of claim 7 , wherein the second sensor patch is determined based on mapping every point of the first sensor patch to the second sensor data.

10. The computer implemented method of claim 7 ,

wherein the second sensor patch is determined based on mapping corner points of the first sensor patch to the second sensor data; and

wherein the second sensor patch is determined based on mapping every point of the first sensor patch to the second sensor data.

11. The computer implemented method of claim 1 , wherein the fusion artificial neural network is trained based on manually labelled occupancy grid map data.

12. The computer implemented method of claim 1 , wherein the fusion artificial neural network is trained based on occupancy grid map data determined based on a third sensor.

13. A system comprising computer hardware, the computer hardware being configured to:

acquire first sensor data of a first sensor in a vicinity of a vehicle;

acquire second sensor data of a second sensor in the vicinity of the vehicle;

determine, based on the second sensor data, a segmented image using a segmentation neural network that determines an object class feature for each segment of the segmented image;

determine a first sensor patch of a pre-determined first sensor patch size from the first sensor data, the first sensor patch corresponding to a potential object in the vicinity of the vehicle;

based on the first sensor patch and the segmented image, determine a second sensor patch of a per-determined second sensor patch size from the second sensor data, the second sensor patch corresponding to a location of the potential object;

extract features of the first sensor patch;

extract features of the second sensor patch including an object class feature for the second sensor patch; and

determine an entry of an occupancy map that corresponds to the potential object by fusing the features of the first sensor patch and the second sensor patch using a fusion artificial neural network.

14. The system of claim 13 , further comprising:

the vehicle;

the first sensor; and

the second sensor.

15. The system of claim 13 ,

wherein the first sensor comprises a radar sensor; and

wherein the second sensor comprises a camera.

16. A non-transitory computer-readable medium comprising instructions that when executed configure computer hardware to:

acquire first sensor data of a first sensor in a vicinity of a vehicle;

acquire second sensor data of a second sensor in the vicinity of the vehicle;

determine, based on the second sensor data, a segmented image using a segmentation neural network that determines an object class feature for each segment of the segmented image;

determine a first sensor patch of a pre-determined first sensor patch size from the first sensor data, the first sensor patch corresponding to a potential object in the vicinity of the vehicle;

based on the first sensor patch and the segmented image, determine a second sensor patch of a pre-determined second sensor patch size from the second sensor data, the second sensor patch corresponding to a location of the potential object;

extract features of the first sensor patch;

extract features of the second sensor patch including an object class feature for the second sensor patch; and

determine an entry of an occupancy map that corresponds to the potential object by fusing the features of the first sensor patch and the second sensor patch using a fusion artificial neural network.

17. The computer implemented method of claim 1 , wherein determining the first sensor patch comprises cutting the first sensor patch out of the first sensor data, the cutting based on centering the first sensor patch of the pre-determined first sensor patch size around a location of the potential object.

18. The computer implemented method of claim 1 , wherein the location of the potential object comprises a radar detection.

19. The system of claim 13 , wherein the first sensor patch and the second sensor patch are multi-dimensional with identical size in at least one dimension.

20. The system of claim 19 , wherein:

the first sensor patch and the second sensor patch have dimensions related to time, width, height, and the features; and

the first sensor patch and the second sensor patch have identical size in the dimensions related to width and height.