Method, apparatus, and computer program product for generating an overhead view of an environment from a perspective image

1. An apparatus comprising at least one processor and at least one non-transitory memory including computer program code instructions, the computer program code instructions configured to, when executed, cause the apparatus to at least:

receive first sensor data from a first vehicle traveling along a road segment in an environment, wherein the first sensor data comprises perspective image data of the environment and wherein the first sensor data comprises a location and a heading;

retrieve a satellite image associated with the location and heading;

apply a deep neural network to regress a bird's eye view image from the perspective image data;

apply a Generative Adversarial Network (GAN) to the regressed bird's eye view image using the satellite image as a target of the GAN to obtain a stabilized bird's eye view image;

derive values of a homography matrix between the sensor data and the stabilized bird's eye view image; and

store the derived values of the homography matrix.

2. The apparatus of claim 1 , wherein the apparatus is further caused to:

receive second sensor data from a second vehicle traveling along a road segment in an environment, wherein the sensor data comprises perspective image data of the environment from a perspective of the second vehicle;

apply a homography matrix including the stored, derived values to the sensor data in substantially real time to warp the perspective image data into a ground plane along which the vehicle is traveling;

generate a bird's eye view of the environment of the vehicle from the perspective image data; and

provide for presentation of the bird's eye view of the environment on a display.

3. The apparatus of claim 2 , wherein the apparatus is further caused to augment satellite image data of the environment with the bird's eye view of the environment.

4. The apparatus of claim 3 , wherein the bird's eye view of the environment includes dynamic objects not found in a satellite image of the environment.

5. The apparatus of claim 1 , wherein causing the apparatus to retrieve a satellite image associated with the location and heading comprises causing the apparatus to identify the location in a satellite image map database and to identify a portion of the satellite image corresponding to the heading.

6. The apparatus of claim 1 , wherein the apparatus is further caused to receive second sensor data comprising a perspective image of an environment and warp the perspective image with the homography matrix including the derived values to obtain a bird's eye view image.

7. The apparatus of claim 6 , wherein the apparatus is further caused to repair a satellite image using the obtained bird's eye view image.

8. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code portions stored therein, the computer-executable program code portions comprising program code instructions configured to:

receive first sensor data from a first vehicle traveling along a road segment in an environment, wherein the first sensor data comprises perspective image data of the environment and wherein the first sensor data comprises a location and a heading;

retrieve a satellite image associated with the location and heading;

apply a deep neural network to regress a bird's eye view image from the perspective image data;

apply a Generative Adversarial Network (GAN) to the regressed bird's eye view image using the satellite image as a target of the GAN to obtain a stabilized bird's eye view image;

derive values of a homography matrix between the sensor data and the stabilized bird's eye view image; and

store the derived values of the homography matrix.

9. The computer program product of claim 1 , further comprising program code instructions to:

receive second sensor data from a second vehicle traveling along a road segment in an environment, wherein the sensor data comprises perspective image data of the environment from a perspective of the second vehicle;

apply a homography matrix including the stored, derived values to the sensor data in substantially real time to warp the perspective image data into a ground plane along which the vehicle is traveling;

generate a bird's eye view of the environment of the vehicle from the perspective image data; and

provide for presentation of the bird's eye view of the environment on a display.

10. The computer program product of claim 9 , further comprising program code instructions to augment satellite image data of the environment with the bird's eye view of the environment.

11. The computer program product of claim 10 , wherein the bird's eye view of the environment includes dynamic objects not found in a satellite image of the environment.

12. The computer program product of claim 8 , wherein the program code instructions to retrieve a satellite image associated with the location and heading comprises program code instructions to identify the location in a satellite image map database and to identify a portion of the satellite image corresponding to the heading.

13. The computer program product of claim 8 , further comprising program code instructions to receive second sensor data comprising a perspective image of an environment and warp the perspective image with the homography matrix including the derived values to obtain a bird's eye view image.

14. The computer program product of claim 13 , further comprising program code instructions to repair a satellite image using the obtained bird's eye view image.

15. A method comprising:

receiving first sensor data from a first vehicle traveling along a road segment in an environment, wherein the first sensor data comprises perspective image data of the environment and wherein the first sensor data comprises a location and a heading;

retrieving a satellite image associated with the location and heading;

applying a deep neural network to regress a bird's eye view image from the perspective image data;

applying a Generative Adversarial Network (GAN) to the regressed bird's eye view image using the satellite image as a target of the GAN to obtain a stabilized bird's eye view image;

deriving values of a homography matrix between the sensor data and the stabilized bird's eye view image; and

storing the derived values of the homography matrix.

16. The method of claim 15 , further comprising:

receiving second sensor data from a second vehicle traveling along a road segment in an environment, wherein the sensor data comprises perspective image data of the environment from a perspective of the second vehicle;

applying a homography matrix including the stored, derived values to the sensor data in substantially real time to warp the perspective image data into a ground plane along which the vehicle is traveling;

generating a bird's eye view of the environment of the vehicle from the perspective image data; and

providing for presentation of the bird's eye view of the environment on a display.

17. The method of claim 16 , further comprising augmenting satellite image data of the environment with the bird's eye view of the environment.

18. The method of claim 17 , wherein the bird's eye view of the environment includes dynamic objects not found in a satellite image of the environment.

19. The method of claim 15 , wherein retrieving a satellite image associated with the location and heading comprises identifying the location in a satellite image map database and identifying a portion of the satellite image corresponding to the heading.

20. The method of claim 15 , further comprising receiving second sensor data comprising a perspective image of an environment and warping the perspective image with the homography matrix including the derived values to obtain a bird's eye view image.