Method, apparatus and computer program for a vehicle

1. A method for a vehicle, the method comprising:

obtaining camera sensor data of a camera of the vehicle;

obtaining Lidar sensor data of a Lidar sensor of the vehicle, the Lidar sensor data comprising a point cloud;

determining information related to a motion of the vehicle;

determining a combined image of the camera sensor data and the Lidar sensor data based on the information related to the motion of the vehicle;

transforming the point cloud within the Lidar sensor data based on the information related to the motion,

determining information related to one or more edges within the camera sensor data and information related to one or more intensity discontinuities within the Lidar sensor data;

determining a mismatch of an overlay of the one or more edges within the camera sensor data and the one or more intensity discontinuities within the Lidar sensor data;

reducing the mismatch of the overlay by transforming at least one of the Lidar sensor data or the camera sensor via shifting, rotating, translating, and/or scaling, and

upsampling the point cloud within the Lidar sensor data of the combined image to match a spatial resolution of color information of the camera sensor data of the combined image, wherein

the determining of the combined image is based on the transformed point cloud.

2. The method according to claim 1 further comprising:

compensating for the egomotion of the vehicle in the determination of the combined image.

3. The method according to claim 1 , wherein

the transforming of the point cloud transposes at least a part of the point cloud to an estimated position of the part of the point cloud at a time a corresponding camera image of the camera sensor data is taken based on the information related to the motion.

4. The method according to claim 1 , further comprising:

performing a time synchronization between the camera sensor data and the Lidar sensor data, wherein the determining of the combined image is further based on the time synchronization between the camera sensor data and the Lidar sensor data.

5. The method according to claim 4 , wherein

the obtaining of the camera sensor data of the camera of the vehicle is triggered based on the Lidar sensor.

6. The method according to claim 5 , wherein

the triggering of the obtaining of the camera sensor data is based on a rotation angle of the Lidar sensor.

7. The method according to claim 6 , wherein

the rotation angle of the Lidar sensor corresponds to a rotation angle when the Lidar completes or commences a recording of a point cloud.

8. The method according to claim 1 , wherein

the determining of the combined image comprises assigning a distance value or a depth value to pixels of the combined image.

9. The method according to claim 1 , wherein

the determining of the combined image comprises overlaying depth information of the Lidar sensor data onto an image constructed from the camera sensor data.

10. A computer product comprising a non-transitory computer readable medium having program code for performing the method according to claim 1 , when the program code is executed on a computer, a processor, or in a programmable hardware component.

11. An apparatus for a vehicle, the apparatus comprising:

at least one interface configured to communicate with a camera of the vehicle and to communicate with a Lidar sensor of the vehicle; and

a processing module configured to:

obtain camera sensor data of the camera,

obtain Lidar sensor data of the Lidar sensor, the Lidar sensor data comprising a point cloud;

determine information related to a motion of the vehicle,

determine a combined image of the camera sensor data and the Lidar sensor data based on the information related to the motion of the vehicle;

transform the point cloud within the Lidar sensor data based on the information related to the motion,

determine information related to one or more edges within the camera sensor data and information related to one or more intensity discontinuities within the Lidar sensor data;

determine a mismatch of an overlay of the one or more edges within the camera sensor data and the one or more intensity discontinuities within the Lidar sensor data;

reduce the mismatch of the overlay by transforming at least one of the Lidar sensor data or the camera sensor via shifting, rotating, translating, and/or scaling, and

upsampling the point cloud within the Lidar sensor data of the combined image to match a spatial resolution of color information of the camera sensor data of the combined image, wherein

the determining of the combined image is based on the transformed point cloud.

12. A vehicle comprising an apparatus according to claim 11 .

13. A method for a vehicle, the method comprising:

obtaining camera sensor data of a camera of the vehicle;

obtaining Lidar sensor data of a Lidar sensor of the vehicle, the Lidar sensor data comprising a point cloud;

determining information related to a motion of the vehicle;

determining a combined image of the camera sensor data and the Lidar sensor data based on the information related to the motion of the vehicle;

transforming the point cloud within the Lidar sensor data based on the information related to the motion, and

upsampling the point cloud within the Lidar sensor data of the combined image to match a spatial resolution of color information of the camera sensor data of the combined image, wherein

the determining of the combined image is based on the transformed point cloud.

14. The method according to claim 13 , wherein the upsampling further comprises:

determining a new distance value for each pixel of color information of the combined information, wherein

the new distance value is not found in the Lidar sensor data.

15. The method according to claim 5 , wherein

the triggering of the obtaining of the camera sensor data occurs when a direction, of a plurality of directions through which the Lidar sensor is rotating, matches a direction that the camera sensor is facing.

16. The method according to claim 13 , further comprising:

determining information related to one or more edges within the camera sensor data and information related to one or more intensity discontinuities within the Lidar sensor data;

determining a mismatch of an overlay of the one or more edges within the camera sensor data and the one or more intensity discontinuities within the Lidar sensor data; and

reducing the mismatch of the overlay by transforming at least one of the Lidar sensor data or the camera sensor via shifting, rotating, translating, and/or scaling.

17. The method according to claim 16 , further comprising:

determining a calibration of the camera sensor data and of the Lidar sensor data based on the information related to the one or more edges and based on the information related to the one or more intensity discontinuities, wherein the determining of the combined image is based on the calibration of the camera sensor data and of the Lidar sensor data.

18. The method according to claim 16 , wherein

the determining of the information related the one or more edges and of the information related to the one or more intensity discontinuities is performed while the vehicle is in motion.

19. The method according to claim 17 , wherein

the determining of the information related the one or more edges and of the information related to the one or more intensity discontinuities is performed while the vehicle is in motion.