Granted Patent B2

US 12,315,097 · App. 18/077,200 · Granted May 27, 2025

Inventors: Miguel Andres Granados Velasquez (Thalwil, CH); Javier Victorio Gomez Gonzalez (Zurich, CH); Danying Hu (Sunnyvale, CA); Eran Guendelman (Tel Aviv, IL); Ali Shahrokni (San Jose, CA); Ashwin Swaminathan (Dublin, CA); Mukta Prasad (Uitikon Waldegg, CH)

Assignee: Magic Leap, Inc.

G06T19/20G06T7/38G06T15/005G06T19/006G06T2219/2004

Abstract

Claims (39)

1. A method of computing a pose between a first component and a map, wherein a pose of the first component is represented by a plurality of collections of features posed with respect to a coordinate frame of the first component, the method comprising:

computing a plurality of candidate localizations based on respective collections of features of the plurality of collections of features; and

determining a localization based on the plurality of candidate localizations, wherein computing the plurality of candidate localizations comprises:

computing a first rough localization for a first collection of features of the plurality of collections of features;

computing, as a first candidate localization of the plurality of candidate localizations, a first refined localization based on the first rough localization;

evaluating one or more criteria;

determining, based on the evaluation of the one or more criteria, whether to perform a second rough localization for a second collection of features of the plurality of collection of features;

when it is determined to not perform the second rough localization for the second collection of features, compute a second refined localization as a second candidate localization of the plurality of candidate localizations based on the first rough localization; and

when it is determined to perform the second rough localization for the second collection of features, compute the second rough localization for the second collection of features and compute the second refined localization as the second candidate localization based on the second rough localization.

2. The method of claim 1 , wherein evaluating the one or more criteria includes computing a confidence metric for the first candidate localization.

3. The method of claim 2 , wherein the confidence metric for the first candidate localization is computed based on the first collection of features a corresponding set of features in a persisted map.

4. The method of claim 3 , wherein the confidence metric for the first candidate localization is computed at least in part by determining a number of features in the first collection of features that are coincident with features of the set of features in the persisted map.

5. The method of claim 1 , wherein evaluating the one or more criteria includes computing a measure of parallax determined between the first collection of features and the second collection of features.

6. The method of claim 1 , wherein evaluating the one or more criteria includes computing a measure of parallax determined based on motion of a portable device.

7. The method of claim 1 , wherein evaluating the one or more criteria includes computing a measure of consensus among the plurality of candidate localizations.

8. The method of claim 7 , comprising computing the second rough localization when the measure of consensus among the plurality of candidate localizations is below a threshold.

9. The method of claim 1 , wherein the second refined localization comprises a localization transformation between a local coordinate frame and a coordinate frame of a persisted map.

10. The method of claim 1 , wherein the plurality of collections of features each comprises a posed feature rig, wherein the posed feature rig comprises a plurality of feature descriptors and a pose associated with each of the plurality of features descriptors.

11. The method of claim 1 , wherein computing the first rough localization comprises selecting a sub-region of a persistent map based on location metadata associated with the first portion of the plurality of collections of features.

12. At least one non-transitory computer readable medium comprising instructions that, when executed by at least one processor, perform a method of computing a pose between a first component and a map, wherein a pose of the first component is represented by a plurality of collections of features posed with respect to a coordinate frame of the first component, the method comprising:

computing a plurality of candidate localizations based on respective collections of features of the plurality of collections of features; and

determining a localization based on the plurality of candidate localizations, wherein computing the plurality of candidate localizations comprises:

computing a first rough localization for a first collection of features of the plurality of collections of features;

computing, as a first candidate localization of the plurality of candidate localizations, a first refined localization based on the first rough localization;

evaluating one or more criteria;

determining, based on the evaluated one or more criteria, whether to perform a second rough localization for a second collection of features of the plurality of collections of features;

when it is determined to not perform the second rough localization for the second collection of features, compute a second refined localization as a second candidate localization of the plurality of candidate localizations based on the first rough localization; and

when it is determined to perform the second rough localization for the second collection of features, compute the second rough localization for the second collection of features and compute the second refined localization as the second candidate localization based on the second rough localization.

13. The at least one non-transitory computer readable medium of claim 12 , wherein:

evaluating the one or more criteria includes computing a confidence metric for the first candidate localization, wherein:

the confidence metric for the first candidate localization is computed based on the first collection of features and a corresponding set of features in a persisted map; and

the confidence metric for the first candidate localization is computed at least in part by determining a number of features in the first collection of features that are coincident with features of the set of features in the persisted map.

14. The at least one non-transitory computer readable medium of claim 12 , wherein evaluating the one or more criteria includes computing a measure of parallax determined between the first collection of features and the second collection of features.

15. The at least one non-transitory computer readable medium of claim 12 , wherein evaluating the one or more criteria includes computing a measure of parallax determined based on motion of a portable device.

16. The at least one non-transitory computer readable medium of claim 12 , wherein evaluating the one or more criteria includes computing a measure of consensus among the plurality of candidate localizations.

17. The at least one non-transitory computer readable medium of claim 16 , wherein the localization component is further configured to compute the second rough localization when the measure of consensus among the plurality of candidate localizations is below a threshold.

18. The at least one non-transitory computer readable medium of claim 12 , wherein the second refined localization comprises a localization transformation between a local coordinate frame and a coordinate frame of a persisted map.

19. The at least one non-transitory computer readable medium of claim 12 , wherein the plurality of collections of features each comprises a posed feature rig, wherein the posed feature rig comprises a plurality of feature descriptors and a pose associated with each of the plurality of features descriptors.

20. The at least one non-transitory computer readable medium of claim 12 , wherein computing the first rough localization comprises selecting a sub-region of a persistent map based on location metadata associated with the first portion of the plurality of collections of features.

Assignments (3)

Continuity (3)

References Cited (400)