IP Library › Granted Patent US 11,361,473
Granted Patent B1
US 11,361,473 · App. 16/984,862 · Granted Jun 14, 2022

Including a physical object based on context

Inventors: Golnaz Abdollahian (San Francisco, CA); Krishna Dayanidhi (Mountain View, CA); Patrick Dillon (Sunnyvale, CA); Aaron R. Zinman (Cupertino, CA)
Assignee: Apple Inc.
G06T11/00G06F3/017G06F3/167G06T7/70G10L15/1815G10L15/22
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,361,473
App. No.
16/984,862
Granted
Jun 14, 2022
Kind
B1
Abstract

The present disclosure relates to techniques for techniques for including a physical object in a computer-generated reality environment. A context associated with the computer-generated reality environment is identified and a physical object located in a real environment is detected. If the detected physical object is associated with the context of the computer-generated reality environment, then the computer-generated reality environment is displayed with a representation of the physical object inserted into the computer-generated reality environment.

Claims (45)

1. An electronic device, comprising:

a display device;

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

displaying, via the display device, a computer-generated reality environment in a first state, wherein the display device is located in a real environment with a physical object, and wherein the computer-generated reality environment in the first state is displayed without a representation of the physical object;

identifying a context associated with the computer-generated reality environment;

detecting, via one or more sensors of the electronic device, the physical object located in the real environment;

in accordance with a determination that the detected physical object is associated with the context of the computer-generated reality environment, displaying, via the display device, the computer-generated reality environment in a second state with a representation of the physical object inserted into the computer-generated reality environment; and

in accordance with a determination that the detected physical object is not associated with the context of the computer-generated reality environment, displaying, via the display device, the computer-generated reality environment in the first state without the representation of the physical object.

2. The electronic device of claim 1 , wherein the determination that the physical object is associated with the context of the computer-generated reality environment is based on a user gesture.

3. The electronic device of claim 1 , wherein the determination that the physical object is associated with the context of the computer-generated reality environment is based on user speech.

4. The electronic device of claim 1 , wherein identifying the context associated with the computer-generated reality environment includes identifying participants in the computer-generated reality environment.

5. The electronic device of claim 1 , wherein identifying the context associated with the computer-generated reality environment includes identifying an application being utilized at the electronic device.

6. The electronic device of claim 1 , wherein identifying the context associated with the computer-generated reality environment includes detecting a conversation topic.

7. The electronic device of claim 1 , wherein identifying the context associated with the computer-generated reality environment includes detecting an activity.

8. The electronic device of claim 1 , wherein the context associated with the computer-generated reality environment is based on historical information.

9. The electronic device of claim 1 , wherein the one or more programs further include instructions for:

in accordance with the determination that the physical object is associated with the context of the computer-generated reality environment and prior to displaying the computer-generated reality environment in the second state with the representation of the physical object, providing a notification that the representation of the physical object is to be included in the computer-generated reality environment.

10. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with a display device, the one or more programs including instructions for:

displaying, via the display device, a computer-generated reality environment in a first state, wherein the display device is located in a real environment with a physical object, and wherein the computer-generated reality environment in the first state is displayed without a representation of the physical object;

identifying a context associated with the computer-generated reality environment;

detecting, via one or more sensors of the electronic device, the physical object located in the real environment;

in accordance with a determination that the detected physical object is associated with the context of the computer-generated reality environment, displaying, via the display device, the computer-generated reality environment in a second state with a representation of the physical object inserted into the computer-generated reality environment; and

in accordance with a determination that the detected physical object is not associated with the context of the computer-generated reality environment, displaying, via the display device, the computer-generated reality environment in the first state without the representation of the physical object.

11. The non-transitory computer-readable storage medium of claim 10 , wherein the determination that the physical object is associated with the context of the computer-generated reality environment is based on a user gesture.

12. The non-transitory computer-readable storage medium of claim 10 , wherein the determination that the physical object is associated with the context of the computer-generated reality environment is based on user speech.

13. The non-transitory computer-readable storage medium of claim 10 , wherein identifying the context associated with the computer-generated reality environment includes identifying participants in the computer-generated reality environment.

14. The non-transitory computer-readable storage medium of claim 10 , wherein identifying the context associated with the computer-generated reality environment includes identifying an application being utilized at the electronic device.

15. The non-transitory computer-readable storage medium of claim 10 , wherein identifying the context associated with the computer-generated reality environment includes detecting a conversation topic.

16. The non-transitory computer-readable storage medium of claim 10 , wherein identifying the context associated with the computer-generated reality environment includes detecting an activity.

17. The non-transitory computer-readable storage medium of claim 10 , wherein the context associated with the computer-generated reality environment is based on historical information.

18. A method, comprising:

at an electronic device with a display device:

displaying, via the display device, a computer-generated reality environment in a first state, wherein the display device is located in a real environment with a physical object, and wherein the computer-generated reality environment in the first state is displayed without a representation of the physical object;

identifying a context associated with the computer-generated reality environment;

detecting, via one or more sensors of the electronic device, the physical object located in the real environment;

in accordance with a determination that the detected physical object is associated with the context of the computer-generated reality environment, displaying, via the display device, the computer-generated reality environment in a second state with a representation of the physical object inserted into the computer-generated reality environment; and

in accordance with a determination that the detected physical object is not associated with the context of the computer-generated reality environment, displaying, via the display device, the computer-generated reality environment in the first state without the representation of the physical object.

19. The method of claim 18 , wherein the determination that the physical object is associated with the context of the computer-generated reality environment is based on a user gesture.

20. The method of claim 18 , wherein the determination that the physical object is associated with the context of the computer-generated reality environment is based on user speech.

21. The method of claim 18 , wherein identifying the context associated with the computer-generated reality environment includes identifying participants in the computer-generated reality environment.

22. The method of claim 18 , wherein identifying the context associated with the computer-generated reality environment includes identifying an application being utilized at the electronic device.

23. The method of claim 18 , wherein identifying the context associated with the computer-generated reality environment includes detecting a conversation topic.

24. The method of claim 18 , wherein identifying the context associated with the computer-generated reality environment includes detecting an activity.

25. The method of claim 18 , wherein the context associated with the computer-generated reality environment is based on historical information.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 25, 2020
From: ABDOLLAHIAN, GOLNAZ; DAYANIDHI, KRISHNA; DILLON, PATRICK; ZINMAN, AARON R.
To: APPLE INC.
Reel/Frame 053594/0322 →
Continuity (1)
Provisional Application 62905951 · Sep 25, 2019
Cited By (7)
US 12,322,052 US 12,417,593 US 12,449,891 US 12,519,924 US 12,524,072 US 12,619,303 US 12,718,494