IP Library › Granted Patent US 12,008,208
Granted Patent B2
US 12,008,208 · App. 18/121,673 · Granted Jun 11, 2024

Merging computer-generated objects

Inventors: Nicolai Georg (San Francisco, CA); Aaron M. Burns (Sunnyvale, CA); Adam G. Poulos (Saratoga, CA); Arun Rakesh Yoganandan (San Francisco, CA); Benjamin Hylak (San Francisco, CA); Benjamin R. Blachnitzky (San Francisco, CA)
G06F3/04815G06F3/014G06F3/0486G06F2203/0331G06F2203/04802
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,008,208
App. No.
18/121,673
Granted
Jun 11, 2024
Kind
B2
Abstract

A method includes displaying a plurality of computer-generated objects, including a first computer-generated object at a first position within an environment and a second computer-generated object at a second position within the environment. The first computer-generated object corresponds to a first user interface element that includes a first set of controls for modifying a content item. The method includes, while displaying the plurality of computer-generated objects, obtaining extremity tracking data. The method includes moving the first computer-generated object from the first position to a third position within the environment based on the extremity tracking data. The method includes, in accordance with a determination that the third position satisfies a proximity threshold with respect to the second position, merging the first computer-generated object with the second computer-generated object in order to generate a third computer-generated object for modifying the content item. The method includes displaying the third computer-generated object.

Claims (51)

1. A method comprising:

at an electronic device with one or more processors, a non-transitory memory, a display, and an extremity tracker:

displaying, on the display, a plurality of computer-generated objects within an extended reality (XR) environment, including a first computer-generated object at a first position within the XR environment, and a second computer-generated object at a second position within the XR environment, wherein the first computer-generated object corresponds to a first user interface element that includes a first set of controls for modifying a content item;

while displaying the plurality of computer-generated objects, obtaining extremity tracking data from the extremity tracker, wherein the extremity tracking data indicates a movement from the first position within the XR environment to a third position within the XR environment;

moving the first computer-generated object from the first position to the third position within the XR environment based on the extremity tracking data; and

in response to moving the first computer-generated object to the third position, in accordance with a determination that the third position satisfies a proximity threshold with respect to the second position:

merging the first computer-generated object with the second computer-generated object in order to generate a third computer-generated object for modifying the content item; and

displaying, on the display, the third computer-generated object within the XR environment.

2. The method of claim 1 , wherein the extremity tracker includes a communication interface provided to communicate with a finger-wearable device, and wherein the extremity tracking data includes finger manipulation data from the finger-wearable device via the communication interface.

3. The method of claim 2 , wherein the finger manipulation data corresponds to sensor data associated with one or more sensors integrated within the finger-wearable device.

4. The method of claim 3 , wherein the sensor data includes positional data output from one or more positional sensors integrated in the finger-wearable device.

5. The method of claim 2 , wherein the extremity tracker further includes a computer vision system, and wherein the computer vision system generates a respective portion of the extremity tracking data by tracking an extremity within image data.

6. The method of claim 3 , wherein the sensor data includes contact intensity data output from a contact intensity sensor integrated in the finger-wearable device.

7. The method of claim 1 , wherein moving the first computer-generated object from the first position to the third position includes selecting the first computer-generated object from the plurality of computer-generated objects based on the extremity tracking data.

8. The method of claim 7 , further comprising determining that the extremity tracking data is indicative of a first gesture, wherein selecting the first computer-generated object is in response to determining that the extremity tracking data is indicative of the first gesture.

9. The method of claim 8 , wherein the first gesture corresponds to a pinch gesture.

10. The method of claim 1 , wherein merging the first computer-generated object with the second computer-generated object is further in response to determining that the extremity tracking data is indicative of a de-pinch gesture.

11. The method of claim 1 , wherein the plurality of computer-generated objects includes a combination of two-dimensional (2D) and three-dimensional (3D) computer-generated objects.

12. The method of claim 1 , wherein merging the first computer-generated object with the second computer-generated object includes ceasing to display the first computer-generated object and the second computer-generated object.

13. The method of claim 1 , further comprising:

determining whether or not the first computer-generated object and the second computer-generated object together satisfy one or more compatibility criteria, wherein merging the first computer-generated object with the second computer-generated object is further in response to determining that the first computer-generated object and the second computer-generated object together satisfy the one or more compatibility criteria; and

in response to determining that the first computer-generated object and the second computer-generated object together do not satisfy the one or more compatibility criteria, foregoing merging the first computer-generated object with the second computer-generated object.

14. The method of claim 13 , wherein the one or more compatibility criteria includes a common content type criterion that is satisfied when the first and second computer-generated objects are associated with a common content type.

15. The method of claim 1 , wherein the second computer-generated object corresponds to a second user interface element that includes a second set of controls for modifying the content item, and wherein the third computer-generated object includes a portion of the first set of controls and a portion of the second set of controls.

16. The method of claim 15 , further comprising:

receiving an input that is directed to within the third computer-generated object, wherein the input selects the portion of the first set of controls within the third computer-generated object; and

in response to receiving the input:

manipulating the portion of the first set of controls based on the input;

manipulating the portion of the second set of controls based on the input; and

modifying the content item according to the manipulations.

17. The method of claim 1 , wherein the second computer-generated object corresponds to the content item.

18. The method of claim 1 , wherein the electronic device corresponds to a head-mountable device (HMD).

19. An electronic device comprising:

one or more processors;

a non-transitory memory;

a display;

an extremity tracker; and

one or more programs, wherein the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

displaying, on the display, a plurality of computer-generated objects, including a first computer-generated object at a first position within an XR environment, and a second computer-generated object at a second position within the XR environment, wherein the first computer-generated object corresponds to a first user interface element that includes a first set of controls for modifying a content item;

while displaying the plurality of computer-generated objects, obtaining extremity tracking data from the extremity tracker, wherein the extremity tracking data indicates a movement from the first position within the XR environment to a third position within the XR environment;

moving the first computer-generated object from the first position to the third position within the XR environment based on the extremity tracking data; and

in response to moving the first computer-generated object to the third position, in accordance with a determination that the third position satisfies a proximity threshold with respect to the second position:

merging the first computer-generated object with the second computer-generated object in order to generate a third computer-generated object for modifying the content item; and

displaying, on the display, the third computer-generated object within the XR environment.

20. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device with one or more processors, a display, and an extremity tracker, cause the electronic device to:

display, on the display, a plurality of computer-generated objects, including a first computer-generated object at a first position within an XR environment, and a second computer-generated object at a second position within the XR environment, wherein the first computer-generated object corresponds to a first user interface element that includes a first set of controls for modifying a content item;

while displaying the plurality of computer-generated objects, obtain extremity tracking data from the extremity tracker, wherein the extremity tracking data indicates a movement from the first position within the XR environment to a third position within the XR environment;

move the first computer-generated object from the first position to the third position within the XR environment based on the extremity tracking data; and

in response to moving the first computer-generated object to the third position, in accordance with a determination that the third position satisfies a proximity threshold with respect to the second position:

merge the first computer-generated object with the second computer-generated object in order to generate a third computer-generated object for modifying the content item; and

display, on the display, the third computer-generated object within the XR environment.

Continuity (3)
Continuation PCTUS2021047878 · Aug 27, 2021
Provisional Application 63079222 · Sep 16, 2020
Related Publication 20230325047A1 · Oct 12, 2023