Cross reality system with fast localization
A cross reality system enables any of multiple devices to efficiently and accurately access previously persisted maps, even maps of very large environments, and render virtual content specified in relation to those maps. The cross reality system may quickly process a batch of images acquired with a portable device to determine whether there is sufficient consistency across the batch in the computed localization. Processing on at least one image from the batch may determine a rough localization of the device to the map. This rough localization result may be used in a refined localization process for the image for which it was generated. The rough localization result may also be selectively propagated to a refined localization process for other images in the batch, enabling rough localization processing to be skipped for the other images.
1. A method of computing a pose between a first component and a map, wherein a pose of the first component is represented by a plurality of collections of features posed with respect to a coordinate frame of the first component, the method comprising:
computing a plurality of candidate localizations based on respective collections of features of the plurality of collections of features; and
determining a localization based on the plurality of candidate localizations, wherein computing the plurality of candidate localizations comprises:
computing a first rough localization for a first collection of features of the plurality of collections of features;
computing, as a first candidate localization of the plurality of candidate localizations, a first refined localization based on the first rough localization;
evaluating one or more criteria;
determining, based on the evaluation of the one or more criteria, whether to perform a second rough localization for a second collection of features of the plurality of collection of features;
when it is determined to not perform the second rough localization for the second collection of features, compute a second refined localization as a second candidate localization of the plurality of candidate localizations based on the first rough localization; and
when it is determined to perform the second rough localization for the second collection of features, compute the second rough localization for the second collection of features and compute the second refined localization as the second candidate localization based on the second rough localization.
2. The method of claim 1 , wherein evaluating the one or more criteria includes computing a confidence metric for the first candidate localization.
3. The method of claim 2 , wherein the confidence metric for the first candidate localization is computed based on the first collection of features a corresponding set of features in a persisted map.
4. The method of claim 3 , wherein the confidence metric for the first candidate localization is computed at least in part by determining a number of features in the first collection of features that are coincident with features of the set of features in the persisted map.
5. The method of claim 1 , wherein evaluating the one or more criteria includes computing a measure of parallax determined between the first collection of features and the second collection of features.
6. The method of claim 1 , wherein evaluating the one or more criteria includes computing a measure of parallax determined based on motion of a portable device.
7. The method of claim 1 , wherein evaluating the one or more criteria includes computing a measure of consensus among the plurality of candidate localizations.
8. The method of claim 7 , comprising computing the second rough localization when the measure of consensus among the plurality of candidate localizations is below a threshold.
9. The method of claim 1 , wherein the second refined localization comprises a localization transformation between a local coordinate frame and a coordinate frame of a persisted map.
10. The method of claim 1 , wherein the plurality of collections of features each comprises a posed feature rig, wherein the posed feature rig comprises a plurality of feature descriptors and a pose associated with each of the plurality of features descriptors.
11. The method of claim 1 , wherein computing the first rough localization comprises selecting a sub-region of a persistent map based on location metadata associated with the first portion of the plurality of collections of features.
12. At least one non-transitory computer readable medium comprising instructions that, when executed by at least one processor, perform a method of computing a pose between a first component and a map, wherein a pose of the first component is represented by a plurality of collections of features posed with respect to a coordinate frame of the first component, the method comprising:
computing a plurality of candidate localizations based on respective collections of features of the plurality of collections of features; and
determining a localization based on the plurality of candidate localizations, wherein computing the plurality of candidate localizations comprises:
computing a first rough localization for a first collection of features of the plurality of collections of features;
computing, as a first candidate localization of the plurality of candidate localizations, a first refined localization based on the first rough localization;
evaluating one or more criteria;
determining, based on the evaluated one or more criteria, whether to perform a second rough localization for a second collection of features of the plurality of collections of features;
when it is determined to not perform the second rough localization for the second collection of features, compute a second refined localization as a second candidate localization of the plurality of candidate localizations based on the first rough localization; and
when it is determined to perform the second rough localization for the second collection of features, compute the second rough localization for the second collection of features and compute the second refined localization as the second candidate localization based on the second rough localization.
13. The at least one non-transitory computer readable medium of claim 12 , wherein:
evaluating the one or more criteria includes computing a confidence metric for the first candidate localization, wherein:
the confidence metric for the first candidate localization is computed based on the first collection of features and a corresponding set of features in a persisted map; and
the confidence metric for the first candidate localization is computed at least in part by determining a number of features in the first collection of features that are coincident with features of the set of features in the persisted map.
14. The at least one non-transitory computer readable medium of claim 12 , wherein evaluating the one or more criteria includes computing a measure of parallax determined between the first collection of features and the second collection of features.
15. The at least one non-transitory computer readable medium of claim 12 , wherein evaluating the one or more criteria includes computing a measure of parallax determined based on motion of a portable device.
16. The at least one non-transitory computer readable medium of claim 12 , wherein evaluating the one or more criteria includes computing a measure of consensus among the plurality of candidate localizations.
17. The at least one non-transitory computer readable medium of claim 16 , wherein the localization component is further configured to compute the second rough localization when the measure of consensus among the plurality of candidate localizations is below a threshold.
18. The at least one non-transitory computer readable medium of claim 12 , wherein the second refined localization comprises a localization transformation between a local coordinate frame and a coordinate frame of a persisted map.
19. The at least one non-transitory computer readable medium of claim 12 , wherein the plurality of collections of features each comprises a posed feature rig, wherein the posed feature rig comprises a plurality of feature descriptors and a pose associated with each of the plurality of features descriptors.
20. The at least one non-transitory computer readable medium of claim 12 , wherein computing the first rough localization comprises selecting a sub-region of a persistent map based on location metadata associated with the first portion of the plurality of collections of features.