IP Library Granted Patent US 11,783,552
Granted Patent B2
US 11,783,552 · App. 17/557,805 · Granted Oct 10, 2023

Identity-based inclusion/exclusion in a computer-generated reality experience

Inventors: Daniel Ulbricht (Sunnyvale, CA); Amit Kumar K C (Sunnyvale, CA); Angela Blechschmidt (San Jose, CA); Chen-Yu Lee (Sunnyvale, CA); Eshan Verma (Mountain View, CA); Mohammad Haris Baig (San Jose, CA); Tanmay Batra (Mountain View, CA)
Assignee: APPLE INC.
G06T19/006A63F13/212A63F13/825G02B27/017G06F3/011G06F3/0304G06T19/003
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,783,552
App. No.
17/557,805
Granted
Oct 10, 2023
Kind
B2
Abstract

In one implementation, a method of including a person in a CGR experience or excluding the person from the CGR experience is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes, while presenting a CGR experience, capturing an image of scene; detecting, in the image of the scene, a person; and determining an identity of the person. The method includes determining, based on the identity of the person, whether to include the person in the CGR experience or exclude the person from the CGR experience. The method includes presenting the CGR experience based on the determination.

Claims (79)

1. A method comprising:

at an electronic device including one or more processors, non-transitory memory, a scene camera, one or more microphones, one or more speakers, and a display:

while presenting a computer-generated reality (CGR) experience via the display, capturing, via the scene camera, an image of a physical environment;

detecting a person within the image of the physical environment; and

in response to detecting the person within the image of the physical environment:

capturing, via the one or more microphones, audio content from the person;

altering the captured audio content from the person and providing the altered audio content, via the one or more speakers, according to a determination that an identity of the person is included among the list of identities; and

providing the captured audio content from the person, via the one or more speakers, according to a determination that the identity of the person is not included among the list of identities.

2. The method of claim 1 , further comprising:

in response to detecting the person within the image of the physical environment:

presenting, via the display, an avatar of the person within the CGR experience according to the determination that the identity of the person is included among the list of identities, wherein the avatar for the person that is different from an appearance of the person detected within the image of the physical environment; and

presenting, via the display, an unaltered version of the person within the CGR experience according to the determination that the identity of the person is not included among the list of identities, wherein the unaltered version of the person corresponds to the appearance of the person detected within the image of the physical environment.

3. The method of claim 1 , further comprising:

in response to detecting the person within the image of the physical environment:

presenting, via the display, a representation of the person within the CGR experience according to the determination that the identity of the person is included among the list of identities; and

forgoing, via the display, presentation of the representation of the person within the CGR experience according to the determination that the identity of the person is not included among the list of identities.

4. The method of claim 3 , further comprising:

presenting, via the display, an inclusion indicia adjacent to the representation of the person according to the determination that the identity of the person is included among the list of identities.

5. The method of claim 1 , further comprising:

presenting, via the display, data associated with the person according to the determination that the identity of the person is included among the list of identities.

6. The method of claim 1 , wherein detecting the person includes applying an object detecting neural network to the image of the physical environment to generate a pixel set corresponding to a representation of the person and wherein determining the identity of the person is based on the pixel set.

7. The method of claim 1 , further comprising:

determining the identity of the person within the image of the physical environment.

8. The method of claim 7 , wherein determining the identity of the person is based on detected facial features of the person.

9. The method of claim 7 , wherein determining the identity of the person includes applying a classifier to at least a portion of the image of the physical environment.

10. The method of claim 9 , further comprising:

determining whether the classifier generates the identity for the person within the image of the physical environment; and

in accordance with a determination that the classifier cannot generate the identity for the person within the image of the physical environment:

presenting, via the display, a notification prompting a user of the electronic device to provide the identity for the person within the image of the physical environment; and

updating the classifier based on a response to the notification.

11. A device comprising:

a scene camera;

a display;

one or more microphones;

one or more speakers;

one or more processors;

a non-transitory memory; and

one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to:

while presenting a computer-generated reality (CGR) experience via the display, capture, via the scene camera, an image of a physical environment;

detecting a person within the image of the physical environment; and

in response to detecting the person within the image of the physical environment:

capture, via the one or more microphones, audio content from the person;

alter the captured audio content from the person and provide the altered audio content, via the one or more speakers, according to a determination that an identity of the person is included among the list of identities; and

provide the captured audio content from the person, via the one or more speakers, according to a determination that the identity of the person is not included among the list of identities.

12. The device of claim 11 , wherein the one or more programs further cause the device to:

in response to detecting the person within the image of the physical environment:

present, via the display, an avatar of the person within the CGR experience according to the determination that the identity of the person is included among the list of identities, wherein the avatar for the person that is different from an appearance of the person detected within the image of the physical environment; and

present, via the display, an unaltered version of the person within the CGR experience according to the determination that the identity of the person is not included among the list of identities, wherein the unaltered version of the person corresponds to the appearance of the person detected within the image of the physical environment.

13. The device of claim 11 , wherein the one or more programs further cause the device to:

in response to detecting the person within the image of the physical environment:

present, via the display, a representation of the person within the CGR experience according to the determination that the identity of the person is included among the list of identities; and

forgo, via the display, presentation of the representation of the person within the CGR experience according to the determination that the identity of the person is not included among the list of identities.

14. The device of claim 13 , wherein the one or more programs further cause the device to:

present, via the display, an inclusion indicia adjacent to the representation of the person according to the determination that the identity of the person is included among the list of identities.

15. The device of claim 11 , wherein the one or more programs further cause the device to:

present, via the display, data associated with the person according to the determination that the identity of the person is included among the list of identities.

16. The device of claim 11 , wherein the one or more programs further cause the device to:

determine the identity of the person within the image of the physical environment, wherein determining the identity of the person is based on at least one of detecting facial features of the person or applying a classifier to at least a portion of the image of the physical environment.

17. A non-transitory memory storing one or more programs, which, when executed by one or more processors of a device with a scene camera, one or more microphones, one or more speakers, and a display, cause the device to:

while presenting a computer-generated reality (CGR) experience via the display, capture, via the scene camera, an image of a physical environment;

detecting a person within the image of the physical environment; and

in response to detecting the person within the image of the physical environment:

capture, via the one or more microphones, audio content from the person;

alter the captured audio content from the person and provide the altered audio content, via the one or more speakers, according to a determination that an identity of the person is included among the list of identities; and

provide the captured audio content from the person, via the one or more speakers, according to a determination that the identity of the person is not included among the list of identities.

18. The non-transitory memory of claim 17 , wherein the one or more programs further cause the device to:

in response to detecting the person within the image of the physical environment:

present, via the display, an avatar of the person within the CGR experience according to the determination that the identity of the person is included among the list of identities, wherein the avatar for the person that is different from an appearance of the person detected within the image of the physical environment; and

present, via the display, an unaltered version of the person within the CGR experience according to the determination that the identity of the person is not included among the list of identities, wherein the unaltered version of the person corresponds to the appearance of the person detected within the image of the physical environment.

19. The non-transitory memory of claim 17 , wherein the one or more programs further cause the device to:

in response to detecting the person within the image of the physical environment:

present, via the display, a representation of the person within the CGR experience according to the determination that the identity of the person is included among the list of identities; and

forgo, via the display, presentation of the representation of the person within the CGR experience according to the determination that the identity of the person is not included among the list of identities.

20. The non-transitory memory of claim 19 , wherein the one or more programs further cause the device to:

present, via the display, an inclusion indicia adjacent to the representation of the person according to the determination that the identity of the person is included among the list of identities.

21. The non-transitory memory of claim 17 , wherein the one or more programs further cause the device to:

present, via the display, data associated with the person according to the determination that the identity of the person is included among the list of identities.

22. The non-transitory memory of claim 17 , wherein the one or more programs further cause the device to:

determine the identity of the person within the image of the physical environment, wherein determining the identity of the person is based on at least one of detecting facial features of the person or applying a classifier to at least a portion of the image of the physical environment.

Continuity (4)
Continuation 17149949 · Jan 15, 2021
Continuation 16580176 · Sep 24, 2019
Provisional Application 62737441 · Sep 27, 2018
Related Publication 20220114796A1 · Apr 14, 2022
Cited By (1)
US 12,536,740