Localization processing service

1 . A method of localizing a mobile subsystem comprising an image sensor component, an orientation sensor component, a memory component, and a processing module communicatively coupled to the image sensor component, the orientation sensor component, and the memory component, the method comprising:

storing, with the memory component, a map feature database comprising a plurality of map feature entries, wherein:

each map feature entry of the plurality of map feature entries is respectively associated with a rendered map image of a plurality of rendered map images rendered from a georeferenced three-dimensional map; and

each map feature entry of the plurality of map feature entries comprises at least one map feature vector indicative of at least one map feature that has been extracted from the rendered map image associated with the map feature entry; and

after the storing, while the mobile subsystem is denied communication with any navigation subsystem, the method further comprises:

moving the mobile subsystem;

after at least a portion of the moving, capturing, at a moment in time with the image sensor component, an image;

extracting, with the processing module, at least one captured image feature from the captured image;

generating, with the processing module, at least one captured image feature vector based on at least one of the at least one extracted captured image feature;

comparing, with the processing module, the at least one captured image feature vector with at least one map feature vector from each map feature entry of at least a portion of the plurality of map feature entries of the stored map feature database;

classifying, with the processing module, at least one particular map feature entry of the plurality of map feature entries as a matching map feature entry based on the comparing; and

defining, with the processing module, an estimated location of the mobile subsystem at the moment in time based on the classifying.

2 . The method of claim 1 , further comprising presenting, substantially in real-time with the capturing, the estimated location of the mobile subsystem to a user of the mobile subsystem.

3 . The method of claim 1 , wherein:

each map feature entry of the plurality of map feature entries further comprises map orientation data indicative of a map orientation of the rendered map image associated with the map feature entry;

the method further comprising:

capturing, at the moment in time with the orientation sensor component, image orientation data indicative of an image sensor orientation of the image sensor component; and

identifying, with the processing module, a proper subset of the plurality of map feature entries based on the image sensor orientation of the image sensor component indicated by the captured image orientation data; and

the comparing comprises comparing the at least one captured image feature vector with at least one map feature vector from each map feature entry of only the proper subset of the plurality of map feature entries.

4 . The method of claim 3 , wherein the map orientation indicated by the map orientation data of each map feature entry of the proper subset of map feature entries is aligned with the image sensor orientation of the image sensor component indicated by the captured image orientation data.

5 . The method of claim 3 , wherein:

each map feature entry of the plurality of map feature entries further comprises map location data indicative of a map location of the rendered map image associated with the map feature entry; and

the defining comprises defining the estimated location of the mobile subsystem at the moment in time based on the map location data of each classified matching map feature entry.

6 . The method of claim 1 , wherein:

each map feature entry of the plurality of map feature entries further comprises map location data indicative of a map location of the rendered map image associated with the map feature entry; and

the defining comprises defining the estimated location of the mobile subsystem at the moment in time based on the map location data of each classified matching map feature entry.

7 . The method of claim 1 , wherein:

each map feature entry of the plurality of map feature entries further comprises map location data indicative of a map location of the rendered map image associated with the map feature entry of the plurality of map feature entries;

the storing further comprises storing, with the memory component, another map feature database comprising another plurality of map feature entries;

each map feature entry of the other plurality of map feature entries is respectively associated with a rendered map image of the plurality of rendered map images rendered from the georeferenced three-dimensional map;

each map feature entry of the other plurality of map feature entries comprises at least one other map feature vector indicative of at least one other map feature that has been extracted from the rendered map image associated with the map feature entry of the other plurality of map feature entries;

each map feature entry of the other plurality of map feature entries further comprises the map location data indicative of the map location of the rendered map image associated with the map feature entry of the other plurality of map feature entries; and

after the classifying but before the defining, the method further comprises identifying, with the processing module, a proper subset of the other plurality of map feature entries based on the map location data of at least one classified matching map feature entry.

8 . The method of claim 7 , wherein the defining comprises defining the estimated location of the mobile subsystem at the moment in time based on the map location data of at least one map feature entry of the proper subset of the other plurality of map feature entries.

9 . The method of claim 7 , further comprising:

extracting, with the processing module, at least one other captured image feature from the captured image;

generating, with the processing module, at least one other captured image feature vector based on at least one of the at least one extracted other captured image feature;

analyzing, with the processing module, the at least one other captured image feature vector in comparison to at least one other map feature vector from each map feature entry of the proper subset of the other plurality of map feature entries of the stored other map feature database; and

categorizing, with the processing module, at least one particular map feature entry of the proper subset of the other plurality of map feature entries as another matching map feature entry based on the analyzing, wherein the defining comprises defining the estimated location of the mobile subsystem at the moment in time based on the categorizing.

10 . The method of claim 9 , wherein:

the at least one other captured image feature is a local image feature; and

the at least one captured image feature is a global image feature.

11 . The method of claim 10 , wherein:

the at least one extracted other captured image feature is an image edges feature; and

the at least one map feature vector of each map feature entry of the other plurality of map feature entries is indicative of at least one map edges feature that has been extracted from the rendered map image associated with the map feature entry of the other plurality of map feature entries.

12 . The method of claim 9 , wherein the defining comprises defining the estimated location of the mobile subsystem at the moment in time based on the map location data of each categorized other matching map feature entry.

13 . The method of claim 1 , wherein:

the at least one extracted captured image feature is an image point feature; and

the at least one map feature vector of each map feature entry of the plurality of map feature entries is indicative of at least one map point feature that has been extracted from the rendered map image associated with the map feature entry.

14 . The method of claim 1 , further comprising rendering the captured image prior to the extracting, wherein the extracting comprises extracting, with the processing module, the at least one captured image feature from the rendered captured image.

15 . The method of claim 1 , further comprising, prior to the comparing:

determining, with the processing module, a last known positioning of the mobile subsystem for the moment in time;

identifying, with the processing module, a subset of the plurality of map feature entries based on the determined last known positioning of the mobile subsystem;

for each map feature entry of the identified subset of the plurality of map feature entries, rendering, with the processing module, from the georeferenced three-dimensional map, the map image associated with that map feature entry; and

extracting, with the processing module, the at least one map feature vector from the rendered map image for each map feature entry of the identified subset of the plurality of map feature entries.

16 . The method of claim 15 , wherein the comparing comprises comparing, with the processing module, the at least one captured image feature vector with the at least extracted one map feature vector from each map feature entry of at least a portion of the identified subset of the plurality of map feature entries.

17 . The method of claim 1 , wherein each map feature entry of the plurality of map feature entries is respectively associated with a position of the georeferenced three-dimensional map.

18 . The method of claim 17 , further comprising, after the at least a portion of the moving and prior to the comparing:

determining, with the processing module, an initial guess of a position of the mobile subsystem for the moment in time;

identifying, with the processing module, a subset of the plurality of map feature entries based on a similarity between the determined initial guess of the position of the mobile subsystem and the position associated with each map feature entry of the subset;

for each map feature entry of the identified subset of the plurality of map feature entries, rendering, with the processing module, from the georeferenced three-dimensional map, the map image associated with that map feature entry; and

extracting, with the processing module, the at least one map feature vector from the rendered map image for each map feature entry of the identified subset of the plurality of map feature entries.

19 . A method of localizing a mobile subsystem comprising an image sensor component, an orientation sensor component, a memory component, and a processing module communicatively coupled to the image sensor component, the orientation sensor component, and the memory component, the method comprising:

storing, with the memory component, a three-dimensional map;

after the storing, moving the mobile subsystem;

after the moving, capturing, at a moment in time with the image sensor component, an image;

determining, with the processing module, an initial guess of the positioning of the mobile subsystem for the moment in time;

rendering, with the processing module, a plurality of rendered map images from the stored three-dimensional map, wherein each rendered map image of the plurality of rendered map images is associated with a respective map positioning that is related to the determined initial guess of the positioning of the mobile subsystem for the moment in time;

extracting, with the processing module, the following:

at least one image feature from the captured image; and

at least one map feature from each rendered map image of at least a subset of the plurality of rendered map images;

comparing, with the processing module, the at least one extracted image feature with the at least one extracted map feature from each rendered map image of the at least the subset of the plurality of rendered map images;

classifying, with the processing module, at least one rendered map image of the plurality of rendered map images as a matching rendered map image based on the comparing; and

defining, with the processing module, an estimated location of the mobile subsystem at the moment in time based on the classifying.

20 . The method of claim 19 , wherein the defining comprises defining the estimated location of the mobile subsystem at the moment in time based on the map positioning associated with each classified matching rendered map image.

21 . The method of claim 20 , wherein the map positioning of any particular rendered map image comprises at least one of the following:

a map orientation; or

a map location.

22 . The method of claim 20 , wherein the determined initial guess of the positioning comprises at least one of the following:

an orientation; or

a location.

23 . The method of claim 19 , wherein each rendered map image of the plurality of rendered map images has a lower resolution than the captured image.

24 . The method of claim 19 , further comprising rendering the captured image prior to the extracting, wherein the extracting comprises extracting, with the processing module, the at least one image feature from the rendered captured image.

25 . A method of localizing a mobile subsystem comprising an image sensor component, an orientation sensor component, a memory component, and a processing module communicatively coupled to the image sensor component, the orientation sensor component, and the memory component, the method comprising:

defining a map feature database comprising a plurality of map feature entries, wherein:

each map feature entry of the plurality of map feature entries is respectively associated with a rendered map image of a plurality of rendered map images rendered from a georeferenced three-dimensional map; and

each map feature entry of the plurality of map feature entries comprises:

at least one map feature vector indicative of at least one map feature that has been extracted from the rendered map image associated with the map feature entry; and

map location data indicative of a map location of the rendered map image associated with the map feature entry;

storing, with the memory component, the defined map feature database;

capturing, at a moment in time with the image sensor component, an image;

extracting, with the processing module, at least one captured image feature from the captured image;

generating, with the processing module, at least one captured image feature vector based on at least one of the at least one extracted captured image feature;

comparing, with the processing module, the at least one captured image feature vector with at least one map feature vector from each map feature entry of at least a portion of the plurality of map feature entries of the stored map feature database;

classifying, with the processing module, at least one particular map feature entry of the plurality of map feature entries as a matching map feature entry based on the comparing;

defining, with the processing module, an estimated location of the mobile subsystem at the moment in time based on the classifying, wherein the defining comprises defining the estimated location of the mobile subsystem at the moment in time based on the map location data of each classified matching map feature entry;

obtaining, at the moment in time with the orientation sensor component, an orientation of the image sensor component;

determining, with the processing module, mobile subsystem orientation data by virtual inertial odometry processing the obtained orientation of the image sensor component in conjunction with the at least one extracted captured image feature; and

determining, with the processing module, a pose estimation of the mobile subsystem at the moment in time by global trajectory alignment processing the determined mobile subsystem orientation data in conjunction with the defined estimated location of the mobile subsystem at the moment in time.

26 . The method of claim 25 , wherein the determined pose estimation comprises a georeferenced position and orientation.

27 . The method of claim 25 , wherein the determining the pose estimation comprises inserting the determined mobile subsystem orientation data and the defined estimated location as constraints in a graph and using non-linear optimization to determine best alignment of the determined mobile subsystem orientation data and the defined estimated location in the graph.

28 . The method of claim 25 , wherein:

the storing the defined map feature database comprises:

storing, with the memory component, a global map feature database comprising a plurality of global map feature entries, wherein each global map feature entry of the plurality of global map feature entries comprises:

a global map feature that has been extracted from a respective global map image that has been rendered from a first three-dimensional map; and

a global map location of the respective global map image;

storing, with the memory component, a local map feature database comprising a plurality of local map feature entries, wherein each local map feature entry of the plurality of local map feature entries comprises:

a local map feature that has been extracted from a respective local map image that has been rendered from a second three-dimensional map; and

a local map location of the respective local map image;

the extracting comprises:

extracting a global image feature from the captured image; and

extracting a local image feature from the captured image;

the method comprising:

identifying, with the processing module, a proper subset of the plurality of global map feature entries based on a comparison of the global image feature with the global map feature of each of at least some entries of the plurality of global map feature entries;

defining, with the processing module, a geographic range based on the global map location of each global map feature entry of the proper subset of the plurality of global map feature entries; and

identifying, with the processing module, a proper subset of the plurality of local map feature entries based on a comparison of the geographic range with the local map location of each of at least some entries of the plurality of local map feature entries; and

the defining comprises defining the estimated location of the mobile subsystem at the moment in time based on a comparison of the local image feature with the local map feature of each of at least some entries of the proper subset of the plurality of local map feature entries.

29 . The method of claim 28 , wherein the first three-dimensional map is the same as the second three-dimensional map.

30 . The method of claim 28 , wherein the first three-dimensional map is different than the second three-dimensional map.