IP Library › Granted Patent US 12675958

Granted Patent B2

US 12675958 · App. 18/587,739 · Granted Jul 7, 2026

System and method of augmented representation of an electronic device

Inventors: Alexis H. Palangie (Palo Alto, CA); Benjamin Hylak (San Francisco, CA); Aaron M. Burns (Sunnyvale, CA); Nathan Gitter (Cupertino, CA)

Assignee: Apple Inc.

G06T19/006 G06T2200/24

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12675958
App. No.: 18/587,739
Granted: Jul 7, 2026
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Some embodiments of the disclosure are directed to an augmented representation of a first electronic device. A three-dimensional representation of a first electronic device (e.g., an augmented device) is presented using a second electronic device in a three-dimensional environment. The three-dimensional environment includes captured portions of a real-world environment, optionally including the first electronic device. The augmented representation of the first electronic device includes a virtual user interface element representing an extension of the physical display of the first electronic device. The representation of the augmented device includes a display of the augmented user interface. The augmented device is optionally configured to display some or all of the user interfaces operating on the first electronic device. Manipulations of and/or interactions with the augmented representation of the first electronic device are possible.

Claims (70)

1 . A method comprising:

at a first electronic device in communication with one or more displays and one or more input devices:

capturing, via the one or more input devices, at least a portion of a real-world environment including a second electronic device that is displaying, via one or more second displays, a first representation of a user interface of the second electronic device;

presenting, via the one or more displays, a three-dimensional environment including a representation of the captured portion of the real-world environment;

while presenting the three-dimensional environment including the representation of the captured portion of the real-world environment, in accordance with a determination that one or more first criteria are satisfied:

presenting, via the one or more displays, a first affordance associated with a representation of the second electronic device;

while presenting the three-dimensional environment including the representation of the captured portion of the real-world environment and the first affordance, receiving, via the one or more input devices, an input corresponding to a selection of the first affordance; and

presenting, in the three-dimensional environment, a second representation of the user interface of the second electronic device different than the first representation of the user interface, in accordance with the input corresponding to the selection of the first affordance, while maintaining presentation of the captured portion of the real-world environment that includes the first representation of the user interface of the second electronic device that is being displayed via the one or more second displays.

2 . The method of claim 1 , wherein the one or more first criteria include a criterion that is satisfied when the one or more second displays of the second electronic device are visible to a user of the first electronic device.

3 . The method of claim 1 , wherein the one or more first criteria include:

a criterion that is satisfied when the second electronic device is in communication with the first electronic device.

4 . The method of claim 1 , further comprising:

while presenting, in the three-dimensional environment, the second representation of the user interface of the second electronic device, causing the one or more second displays of the second electronic device to power down.

5 . The method of claim 1 , wherein presenting, in the three-dimensional environment, the second representation of the user interface of the second electronic device includes:

presenting, within the second representation of the user interface of the second electronic device, a cursor controllable by one or more input devices in communication with the second electronic device;

wherein presentation of a movement of the cursor is bounded within the second representation of the user interface of the second electronic device.

6 . The method of claim 1 , further comprising:

while presenting, in the three-dimensional environment, the second representation of the user interface of the second electronic device, receiving an input corresponding to a manipulation of contents of the second representation of the user interface of the second electronic device; and

in accordance with the input corresponding to the manipulation of the representation of the user interface, updating an appearance of the second representation of the user interface in accordance with the input.

7 . The method of claim 1 , wherein presenting, in the three-dimensional environment, the second representation of the user interface of the second electronic device also includes:

presenting, in the three-dimensional environment, a first portion of the first representation of the user interface of the second electronic device at a region of the three-dimensional environment at which the one or more second displays are located and a second portion of the first representation of the user interface in a user interface element within a threshold distance of the region of the three-dimensional environment at which the one or more second displays are located.

8 . The method of claim 7 , wherein the second representation of the user interface of the second electronic device includes a cursor controllable by one or more input devices in communication with the second electronic device;

wherein presentation of a movement of the cursor is bounded by the region of the three-dimensional environment at which the one or more second displays are located and the user interface element within the threshold distance of the region of the three-dimensional environment at which the one or more second displays are located.

9 . A first electronic device comprising:

one or more processors;

memory; and

one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing a method comprising:

capturing, via one or more input devices, at least a portion of a real-world environment including a second electronic device that is displaying, via one or more second displays, a first representation of a user interface of the second electronic device;

presenting, via one or more displays, a three-dimensional environment including a representation of the captured portion of the real-world environment;

presenting, via the one or more displays, a first affordance associated with a representation of the second electronic device;

10 . The first electronic device of claim 9 , wherein the one or more first criteria include a criterion that is satisfied when the one or more second displays of the second electronic device are visible to a user of the first electronic device.

11 . The first electronic device of claim 9 , wherein the one or more first criteria include:

a criterion that is satisfied when the second electronic device is in communication with the first electronic device.

12 . The first electronic device of claim 9 , wherein the method further comprises:

13 . The first electronic device of claim 9 , wherein presenting, in the three-dimensional environment, the second representation of the user interface of the second electronic device includes:

wherein presentation of a movement of the cursor is bounded within the second representation of the user interface of the second electronic device.

14 . The first electronic device of claim 9 , wherein the method further comprises:

15 . The first electronic device of claim 9 , wherein presenting, in the three-dimensional environment, the second representation of the user interface of the second electronic device also includes:

16 . The first electronic device of claim 15 , wherein the second representation of the user interface of the second electronic device includes a cursor controllable by one or more input devices in communication with the second electronic device;

17 . A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of a first electronic device, cause the first electronic device to perform a method comprising:

presenting, via one or more displays, a three-dimensional environment including a representation of the captured portion of the real-world environment;

presenting, via the one or more displays, a first affordance associated with a representation of the second electronic device;

18 . The non-transitory computer readable storage medium of claim 17 , wherein the one or more first criteria include a criterion that is satisfied when the one or more second displays of the second electronic device are visible to a user of the first electronic device.

19 . The non-transitory computer readable storage medium of claim 17 , wherein the one or more first criteria include:

a criterion that is satisfied when the second electronic device is in communication with the first electronic device.

20 . The non-transitory computer readable storage medium of claim 17 , wherein the method further comprises:

21 . The non-transitory computer readable storage medium of claim 17 , wherein presenting, in the three-dimensional environment, the second representation of the user interface of the second electronic device includes:

wherein presentation of a movement of the cursor is bounded within the second representation of the user interface of the second electronic device.

22 . The non-transitory computer readable storage medium of claim 17 , wherein the method further comprises:

23 . The non-transitory computer readable storage medium of claim 17 , wherein presenting, in the three-dimensional environment, the second representation of the user interface of the second electronic device also includes:

24 . The non-transitory computer readable storage medium of claim 23 , wherein the second representation of the user interface of the second electronic device includes a cursor controllable by one or more input devices in communication with the second electronic device;

wherein presentation of a movement of the cursor is bounded by the region of the three-dimensional environment at which the one or more second displays-are located and the user interface element within the threshold distance of the region of the three-dimensional environment at which the one or more second displays are located.