Multi-sync ensemble model for device localization
A system and method determine the location of a device. The device collects sensor data using one or more sensors. Based on the sensor data, one or more localization models are selected from a plurality of localization models. The selected models are applied to generate one or more candidate locations. The current location of the device is determined based on the one or more candidate locations.
1. A method for determining a location of a mobile device, the method comprising:
collecting image data captured by a camera on the mobile device;
identifying one or more features from the image data;
determining whether the mobile device is indoors or outdoors based on the identified features;
selecting a localization model from a plurality of localization models based on whether the mobile device is indoors or outdoors;
providing the image data as input to the selected localization model, the selected localization model outputting a candidate location; and
determining the location of the mobile device based on the candidate location.
2. The method of claim 1 , wherein identifying the one or more features from the image data comprises applying an object recognition model to the image data to identify a physical object in an environment captured by the image data as a feature of the one or more features.
3. The method of claim 2 , wherein determining whether the mobile device is indoors or outdoors based on the identified features comprises characterizing the physical object as an indoor object or an outdoor object, wherein the determination is based on the characterization of the physical object.
4. The method of claim 3 , wherein the object recognition model is configured to identify objects including trees, sky, grass, chairs, tables, street lamps, billboards, and street signs.
5. The method of claim 2 , wherein selecting the localization model is further based on the physical object.
6. The method of claim 1 , wherein the plurality of localization models includes a combination of: a point cloud based model, a plane matching model, a line matching model, a geographic information system (GIS) model, a building recognition model, an object recognition model, a semantic matching model, a cube matching model, a cylinder matching model, a horizon matching model, a light source matching model, or a landscape recognition model.
7. The method of claim 1 , further comprising:
selecting one or more additional localization models from the plurality of localization models; and
providing the image data as input to the one or more additional localization models, the one or more additional localization models outputting one or more additional candidate locations,
wherein determining the location of the mobile device is further based on the one or more additional candidate locations.
8. The method of claim 7 , wherein determining the location of the mobile device comprises determining an average location of the candidate location and the one or more additional candidate locations.
9. The method of claim 1 , wherein selecting the localization model is further based on GPS data indicating a current location of the mobile device.
10. A non-transitory computer-readable storage medium comprising instructions that, when executed by a computing device, cause the computing device to perform operations including:
collecting image data captured by a camera on a mobile device;
identifying one or more features from the image data;
determining whether the mobile device is indoors or outdoors based on the identified features;
selecting a localization model from a plurality of localization models based on whether the mobile device is indoors or outdoors;
providing the image data as input to the selected localization model, the selected localization model outputting a candidate location; and
determining a location of the mobile device based on the candidate location.
11. The storage medium of claim 10 , wherein identifying the one or more features from the image data comprises applying an object recognition model to the image data to identify one or more physical objects in an environment captured by the image data as the one or more features.
12. The storage medium of claim 11 , wherein determining whether the mobile device is indoors or outdoors based on the identified features comprises characterizing the physical object as an indoor object or an outdoor object, wherein the determination is based on the characterization of the physical object.
13. The storage medium of claim 12 , wherein the object recognition model is configured to identify objects including trees, sky, grass, chairs, tables, street lamps, billboards, and street signs.
14. The storage medium of claim 11 , wherein selecting the localization model is further based on the physical object.
15. The storage medium of claim 10 , wherein the plurality of localization models includes a combination of: a point cloud based model, a plane matching model, a line matching model, a geographic information system (GIS) model, a building recognition model, an object recognition model, a semantic matching model, a cube matching model, a cylinder matching model, a horizon matching model, a light source matching model, and a landscape recognition model.
16. The storage medium of claim 10 , wherein the operations further comprise:
selecting one or more additional localization models from the plurality of localization models; and
providing the image data as input to the one or more additional localization models, the one or more additional localization models outputting one or more additional candidate locations,
wherein determining the location of the mobile device is further based on the one or more additional candidate locations.
17. The storage medium of claim 16 , wherein determining the location of the mobile device comprises determining an average location of the candidate location and the one or more additional candidate locations.
18. The storage medium of claim 10 , wherein selecting the localization model is further based on GPS data indicating a current location of the mobile device.
19. The storage medium of claim 10 , wherein the computing device is the mobile device.
20. A device comprising:
a processor; and
a non-transitory computer-readable storage medium storing instructions that, when executed by the processor, cause the processor to perform operations including:
collecting image data captured by a camera on a mobile device;
identifying one or more features from the image data;
determining whether the mobile device is indoors or outdoors based on the identified features;
selecting a localization model from a plurality of localization models based on whether the mobile device is indoors or outdoors;
providing the image data as input to the selected localization model, the selected localization model outputting a candidate location; and
determining a location of the mobile device based on the candidate location.