Granted Patent B1

US 11,113,893 · App. 16/950,823 · Granted Sep 7, 2021

Inventors: Jing Ma (Mill Creek, WA); Gerrit Hendrik Hofmeester (Seattle, WA); John Jacob Blakeley (Seattle, WA); Camila Cortes De Almeida e De Vincenzo (Seattle, WA); Gagneet Singh Mac (Pacifica, CA); Jenna Velez (Seattle, WA); Cody Char (Seattle, WA); Annika Rodrigues (Lynnwood, WA); Michael Luo (Seattle, WA)

G06T19/006G02B27/0093G06Q50/01G02B27/017

Abstract

Claims (65)

1. A method for presenting a glint related to an object in an environment depicted by an extended reality (XR) device, the method comprising:

identifying parameters of the object in the environment, the object related to the glint that is linked to a geographic position of the object;

determining that a distance between the geographic position of the object and a location of the XR device is greater than a first threshold distance, and in response:

presenting an indication of the glint on a display area depicting out-of-view objects;

determining that the distance between the geographic position of the object and the location of the XR device has updated to be less than the first threshold distance and greater than a second threshold distance that is less than that of the first threshold distance, and in response:

presenting the glint a threshold amount above the object that is related to the glint; and

determining that the distance between the geographic position of the object and the location of the XR device has updated to be less than the second threshold, and in response:

presenting the glint as attached to the object that is related to the glint

wherein presenting the glint attached to the object includes presenting the glint in a minimized form and wherein, responsive to detecting a first selection action of the glint, the presentation of the minimized glint is updated to a maximized version with A) controls for interacting with the glint and/or B) additional content associated with the glint.

2. The method of claim 1 , wherein presenting the indication of the glint on a display area depicting out-of-view objects comprises:

presenting the glint in a minimized form on a compass portion of the environment displayed on the XR device.

3. The method of claim 1 , wherein the threshold amount, that the glint is displayed above the visible object related to the glint, is an amount determined to make the glint higher than an intervening object that is between the XR device and the object related to the glint.

4. The method of claim 1 , wherein presenting the glint as attached to the object related to the glint comprises:

presenting the glint to be, from a point of view of a user of the XR device, a pre-determined amount to the left or right of the object related to the glint.

5. The method of claim 1 , wherein the first selection action of the glint includes determining that an attention focal point of the XR device is directed at a location of the glint, wherein the focal point is determined by the XR device based on any of eye tracking for a user, head tracking for the user, hand tracking for the user, or any combination thereof.

6. The method of claim 1 , wherein, the maximized version of the glint, includes displaying the controls for interacting with the glint, and wherein the controls provide access to actions for an application relating to the glint, the actions being responsive to detecting a second selection action in relation to the controls.

7. The method of claim 1 ,

wherein the maximized version of the glint, includes displaying the additional content,

wherein the additional content includes an indication of an interaction on a social network in relation to the object, and

wherein the object is a real-world object in the environment.

8. The method of claim 1 , further comprising:

determining that the glint is one of a number of glints within a specified area; and

responsive to determining that the number of glints are within the specified area, updating the environment displayed on the XR device to replace the number of glints with a group glint indicator indicative of the number of glints.

9. The method of claim 1 , further comprising:

identifying the object as a first detected object by comparing detectable features of the object with characteristics of a series of known objects;

processing visual features of the first detected object to derive a series of search results corresponding to content and/or applications relating to the first detected object;

identifying one or more glints, including the glint, based on the search results; and

presenting the one or more glints, in the environment depicted in the XR device, in spatial relation to the object.

10. A system comprising:

one or more processors; and

one or more memories storing instructions that, when executed, cause the one or more processors to perform a process comprising:

identifying parameters of an object in an environment depicted by an extra reality (XR) device, the object related to a glint that is linked to a geographic position of the object;

determining whether a distance between the geographic position of the object and a location of the XR device is greater than any of a first threshold distance and a second threshold distance, wherein:

responsive to determining that the distance is greater than both the first threshold distance and second threshold distance, an indication of the glint is presented on a display area depicting out-of-view objects is presented on the XR device;

responsive to determining that the distance is less than the first threshold distance and greater than the second threshold distance, the glint is presented a threshold amount above the object that is related to the glint; and

responsive to determining that the distance is less than both the first threshold distance and second threshold distance, the glint is presented as attached to the object related to the glint.

11. The system of claim 10 , wherein presenting the glint attached to the object includes presenting the glint in a minimized form and wherein, responsive to detecting a first selection action of the glint, the presentation of the minimized glint is updated to a maximized version with A) controls for interacting with the glint and/or B) additional content associated with the glint.

12. The system of claim 10 , wherein presenting the indication of the glint on a display area depicting out-of-view objects comprises:

presenting the glint in a minimized form on a compass portion of the environment displayed on the XR device.

13. The system of claim 10 , wherein the maximized version of the glint, includes displaying the additional content, wherein the additional content includes an indication of an interaction on a social network in relation to the object, and wherein the object is a real-world object in the environment.

14. The system of claim 10 , wherein the process further comprises:

determining that the glint is one of a number of glints within a specified area; and

responsive to determining that the number of glints are within the specified area, updating the environment displayed on the XR device to replace the number of glints with a group glint indicator indicative of the number of glints.

15. The system of claim 10 , wherein the process further comprises:

identifying visual features of the object;

deriving a series of search results, based on the identified visual features, corresponding to content and/or applications relating to the object;

identifying one or more glints, including the glint, based on the search results; and

presenting the one or more glints, in the environment depicted in the XR device, in spatial relation to the object.

16. A machine-readable medium having machine executable instructions stored thereon that, when executed by one or more processors, direct the one or more processors to perform a method comprising:

determining that a distance between a geographic position of an object and a location of an XR device is greater than a first threshold distance and in response:

presenting an indication of a glint related to the object on a display area depicting out-of-view objects;

determining that the distance between the geographic position of the object and the location of the XR device has updated to be less than the first threshold distance and greater than a second threshold distance that is less than that of the first threshold distance and in response:

presenting the glint a threshold amount above the object that is related to the glint; and

determining that the distance between the geographic position of the object and the location of the XR device has updated to be less than the second threshold and in response:

presenting the glint attached to the object that is related to the glint.

17. The machine-readable medium of claim 16 , wherein presenting the glint attached to the object includes presenting the glint in a minimized form and wherein, responsive to detecting a first selection action of the glint, the presentation of the minimized glint is updated to a maximized version with A) controls for interacting with the glint and/or B) additional content associated with the glint.

18. The machine-readable medium of claim 17 , wherein the first selection action of the glint includes determining that an attention focal point of the XR device is directed at a location of the glint, wherein the focal point is determined by the XR device based on any of eye tracking for a user, head tracking for the user, hand tracking for the user, or any combination thereof.

19. The machine-readable medium of claim 16 , wherein the method further comprises:

determining that the glint is one of a number of glints within a specified area; and

responsive to determining that the number of glints are within the specified area, updating the environment displayed on the XR device to replace the number of glints with a group glint indicator indicative of the number of glints.

20. The machine-readable medium of claim 16 , wherein the method further comprises:

identifying visual features of the object;

processing the visual features of the object to derive a series of search results corresponding to content and/or applications relating to the object;

identifying one or more glints, including the glint, based on the search results; and

presenting the one or more glints, in the environment depicted in the XR device, in spatial relation to the object.

Assignments (2)

Cited By (12)

US 12,197,634 US 12,254,581 US 12,272,012 US 12,292,935 US 12,353,443 US 12,353,673 US 12,548,245 US 12,572,253 US 12,613,938 US 12,633,068 US 12,670,001 US 12,681,561