Unsupervised approach to environment mapping at night using monocular vision

1. A map provider system configured for providing a feature map for vehicle localization under low illumination level conditions, the map provider system comprising:

at least one memory, the at least one memory storing a digital map;

at least one communications interface configured to enable communication with one or more routing and navigation systems;

at least one processor configured to:

operate a trained deep neural network, the trained deep neural network configured to:

receive an image captured under low illumination level conditions and pose data corresponding to the image;

identify a feature within the image;

analyze the image to extract feature information corresponding to the feature from the image;

determine, based on the image and the pose data, geo-location information corresponding to the feature; and

append the geo-location information to the feature information; and

store the feature information as part of a feature map layer of the digital map; and

cause the at least one communications interface to provide at least a portion of the digital map to a routing and navigation system, wherein the routing and navigation system is configured to (a) perform a localization determination based on the feature map layer of the portion of the digital map and (b) make a routing decision based on the localization determination.

2. A map provider system according to claim 1 , wherein, to train the deep neural network, the at least one processor is configured to:

cause the deep neural network to receive a sequence of training images, the sequence of training images captured under particular conditions;

analyze, by the deep neural network, a portion of the sequence of training images, the sequence of training images comprising a particular training image and the portion of the sequence of training images comprising two or more training images of the sequence of training images that are not the particular training image;

identify, by the deep neural network, one or more features based on the analysis of the portion of sequence of training images;

generate, by the deep neural network, an encoded representation of the particular training image;

determine a loss function based on an analysis of the particular training image and the encoded representation of the particular training image; and

update a network weight of the deep neural network based at least in part on the loss function.

3. A map provider system according to claim 2 , wherein the deep neural network is formatted as an auto-encoder representation.

4. A map provider system according to claim 2 , wherein the sequence of training images is a temporal sequence and the particular training image is a middle image of the temporal sequence.

5. A map provider system according to claim 2 , wherein (a) a sequence of pose information is provided to the deep neural network, (b) each instance of the sequence of pose information corresponds to a training image of the sequence of training images, and (c) the one or more features are identified based at least in part on motion indicated by the sequence of pose information.

6. A map provider system according to claim 1 , wherein the image is captured at nighttime.

7. A method comprising:

receiving, by a trained deep neural network, an image captured under low illumination level conditions and pose data corresponding to the image;

identifying, by the trained deep neural network, a feature within the image;

analyzing the image with the trained deep neural network to extract feature information corresponding to the feature from the image;

determining, by the trained deep neural network and based on the image and the pose data, geo-location information corresponding to the feature;

appending, by the deep neural network, the geo-location information to the feature information;

storing the feature information as part of a feature map layer of a digital map; and

providing at least a portion of the digital map to a routing and navigation system.

8. A method according to claim 7 , wherein training the deep neural network comprises:

receiving, by the deep neural network, a sequence of training images, the sequence of training images captured under particular conditions;

analyzing, by the deep neural network, a portion of the sequence of training images, the sequence of training images comprising a particular training image and the portion of the sequence of training images comprising two or more training images of the sequence of training images that are not the particular training image;

identifying, by the deep neural network, one or more features based on the analysis of the portion of sequence of training images;

generating, by the deep neural network, an encoded representation of the particular training image;

determining a loss function based on an analysis of the particular training image and the encoded representation of the particular training image; and

updating a network weight of the deep neural network based at least in part on the loss function.

9. A method according to claim 8 , wherein the deep neural network is formatted as an auto-encoder representation.

10. A method according to claim 8 , wherein the sequence of training images is a temporal sequence and the particular training image is a middle image of the temporal sequence.

11. A method according to claim 8 , wherein (a) a sequence of pose information is provided to the deep neural network, (b) each instance of the sequence of pose information corresponds to a training image of the sequence of training images, and (c) the one or more features are identified based at least in part on motion indicated by the sequence of pose information.

12. A method according to claim 7 , wherein the routing and navigation system performs one or more localization determinations based on information stored in the feature map layer and at least one of (a) performs one or more route planning decisions, (b) operates a vehicle based on the one or more localization determinations, or (c) both.

13. A method according to claim 7 , wherein the image is captured at nighttime.

14. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions configured to:

operate a trained deep neural network to:

receive an image captured under low illumination level conditions and pose data corresponding to the image;

identify a feature within the image;

analyze the image to extract feature information corresponding to the feature from the image;

determine, based on the image and the pose data, geo-location information corresponding to the feature; and

append the geo-location information to the feature information;

store the feature information as part of a feature map layer of a digital map; and

provide at least a portion of the digital map to a routing and navigation system.

15. A computer program product according to claim 14 , the computer-executable program code instructions comprising program code instructions configured to train the deep neural network by:

receiving, by the deep neural network, a sequence of training images, the sequence of training images captured under particular conditions;

analyzing, by the deep neural network, a portion of the sequence of training images, the sequence of training images comprising a particular training image and the portion of the sequence of training images comprising two or more training images of the sequence of training images that are not the particular training image;

identifying, by the deep neural network, one or more features based on the analysis of the portion of sequence of training images;

generating, by the deep neural network, an encoded representation of the particular training image;

determining a loss function based on an analysis of the particular training image and the encoded representation of the particular training image; and

updating a network weight of the deep neural network based at least in part on the loss function.

16. A computer program product according to claim 15 , wherein the deep neural network is formatted as an auto-encoder representation.

17. A computer program product according to claim 15 , wherein the sequence of training images is a temporal sequence and the particular training image is a middle image of the temporal sequence.

18. A computer program product according to claim 15 , wherein (a) a sequence of pose information is provided to the deep neural network, (b) each instance of the sequence of pose information corresponds to a training image of the sequence of training images, and (c) the one or more features are identified based at least in part on motion indicated by the sequence of pose information.

19. A computer program product according to claim 14 , wherein the routing and navigation system performs one or more localization determinations based on information stored in the feature map layer and at least one of (a) performs one or more route planning decisions, (b) operates a vehicle based on the one or more localization determinations, or (c) both.

20. A computer program product according to claim 14 , wherein the image is captured at nighttime.