Granted Patent B2

US 12,417,596 · App. 18/367,418 · Granted Sep 16, 2025

Inventors: Jesse Chand (San Francisco, CA); Kristi E. Bauerly (Los Altos, CA); Stephen O. Lemay (Palo Alto, CA); Jonathan R. Dascola (San Francisco, CA); Shih-Sang Chiu (San Francisco, CA)

Assignee: Apple Inc.

G06T19/006G06T19/20H04N7/152H04N7/157G06T2219/2004

Abstract

Claims (118)

1. A computer system configured to communicate with a display generation component, the computer system comprising:

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

while participating in a communication session that is a spatial communication session that includes displaying, via the display generation component, representations of a plurality of participants in the communication session in a spatially distributed arrangement in a 3D environment, wherein displaying the plurality of participants in the spatially distributed arrangement includes displaying:

the representations of the plurality of participants spaced apart from each other and a user of the computer system by at least a threshold amount in a first non-vertical direction in the 3D environment; and

the representations of the plurality of participants spaced apart from each other and the user by at least the threshold amount in a second non-vertical direction that is different from the first non-vertical direction;

while displaying the representations of the plurality of participants distributed in the 3D environment, detecting an event; and

in response to detecting the event, transitioning the communication session from the spatial communication session to a non-spatial communication session that includes displaying, via the display generation component, representations of at least a subset of the plurality of participants of the communication session in a grouped arrangement, wherein in the grouped arrangement:

the representations of the plurality of participants are spaced apart from each other by less than the threshold amount in the first non-vertical direction in the 3D environment;

a representation of a first participant in the grouped arrangement has a different position than a representation of the first participant in the spatially distributed arrangement; and

a representation of a second participant in the grouped arrangement has a different position than a representation of the second participant in the spatially distributed arrangement.

2. The computer system of claim 1 , wherein:

in the non-spatial communication session:

a representation of a first participant of the plurality of participants is in a first window region, and

a representation of a second participant of the plurality of participants is in a second window region that is different from the first window region; and

in the spatial communication session:

a representation of the first participant of the plurality of participants is not in a window region, and

a representation of the second participant of the plurality of participants is not in a window region.

3. The computer system of claim 2 , wherein the representation of the first participant is a simulated three-dimensional representation and the representation of the second participant is a two-dimensional representation.

4. The computer system of claim 2 , wherein the plurality of participants are two-dimensional representations.

5. The computer system of claim 2 , wherein the plurality of participants are three-dimensional representations.

6. The computer system of claim 1 , wherein the event is a request received during the communication session to transition a representation of the user of the computer system from a 3D representation to a 2D representation.

7. The computer system of claim 6 , wherein the request is based on an input in a communication session control region.

8. The computer system of claim 7 , wherein the communication session control region includes an option to transition the representation of the user of the computer system from the 3D representation to the 2D representation and one or more options corresponding to other communication session controls.

9. The computer system of claim 1 , wherein the event is a request received during the communication session to transition the communication session from the spatial communication session to the non-spatial communication session.

10. The computer system of claim 1 , wherein the event is an additional participant joining the communication session.

11. The computer system of claim 10 , wherein the additional participant joining the communication session causes a number of participants represented by simulated three-dimensional representations to exceed a threshold number of participants.

12. The computer system of claim 1 , the one or more programs further including instructions for:

while the communication session is a non-spatial communication session, shifting a position of a respective window region corresponding to a respective participant based on the respective participant moving.

13. The computer system of claim 12 , wherein the respective window region moves forward and/or backward in a virtual environment based on a head position of respective participant.

14. The computer system of claim 12 , wherein the respective window region tilts based on a head position of respective participant.

15. The computer system of claim 12 , wherein a first window shifts in a first direction based on movement of a participant displayed in the first window and a second window shifts in a second direction, different from the first direction, based on movement of a participant displayed in the second window.

16. The computer system of claim 1 , the one or more programs further including instructions for:

while participating in the communication session that is a non-spatial communication session, detecting a second event; and

in response to detecting the second event, transitioning the communication session from the non-spatial communication session to the spatial communication session.

17. The computer system of claim 16 , wherein the second event is a participant leaving the communication session.

18. The computer system of claim 16 , wherein the second event is a request received during the communication session to transition a representation of the user of the computer system from a 2D representation to a 3D representation.

19. The computer system of claim 16 , wherein the second event is a request received during the communication session to transition the communication session from a non-spatial communication session to a spatial communication session.

20. The computer system of claim 1 , the one or more programs further including instructions for:

while in a spatial communication session, displaying, via the display generation component, a self-view of a representation of the user of the computer system in a self-view window region.

21. The computer system of claim 20 , wherein the self-view window region overlaps a window region that includes a representation of another participant of an ongoing communication session.

22. The computer system of claim 20 , wherein the self-view window region is smaller than a window region that includes a representation of another participant.

23. The computer system of claim 1 , wherein:

during a spatial communication session, a first participant of the communication session is enabled to move a respective representation of the first participant and a second participant of the communication session is enabled to move a respective representation of the second participant; and

during a non-spatial communication session, a user of the computer system is enabled to move respective window regions that include respective representations of the plurality of participants of the communication session.

24. The computer system of claim 23 , wherein the computer system is configured to communicate with one or more sensors, the one or more programs further including instructions for:

detecting, via the one or more sensors, user input to reposition a respective window region that includes a respective representation of a participant; and

in response to detecting the user input to reposition the respective window region that includes the respective representation of the participant, repositioning a plurality of window regions of the plurality of participants.

25. The computer system of claim 23 , wherein the respective representations of the plurality of participants are placed in an initial placement based on predetermined placement rules.

26. The computer system of claim 23 , the one or more programs further including instructions for:

displaying, via the display generation component, a representation of an invited user that is not currently a participant in the communication session;

in accordance with a determination that the communication session is a non-spatial communication session, enabling a user of the computer system to reposition the representation of the invited user that is not currently a participant; and

in accordance with a determination that the communication session is a spatial communication session, forgoing enabling the user of the computer system to reposition the representation of the invited user that is not currently a participant.

27. The computer system of claim 1 , the one or more programs further including instructions for:

in response to changing between a spatial communication and a non-spatial communication session, displaying, via the display generation component, an indication that a mode of the communication session has changed.

28. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that is in communication with a display generation component, the one or more programs including instructions for:

while participating in a communication session that is a spatial communication session that includes displaying, via the display generation component, representations of a plurality of participants in the communication session in a spatially distributed arrangement in a 3D environment, wherein displaying the plurality of participants in the spatially distributed arrangement includes displaying:

the representations of the plurality of participants spaced apart from each other and a user of the computer system by at least a threshold amount in a first non-vertical direction in the 3D environment; and

the representations of the plurality of participants spaced apart from each other and the user by at least the threshold amount in a second non-vertical direction that is different from the first non-vertical direction;

while displaying the representations of the plurality of participants distributed in the 3D environment, detecting an event; and

in response to detecting the event, transitioning the communication session from the spatial communication session to a non-spatial communication session that includes displaying, via the display generation component, representations of at least a subset of the plurality of participants of the communication session in a grouped arrangement, wherein in the grouped arrangement:

the representations of the plurality of participants are spaced apart from each other by less than the threshold amount in the first non-vertical direction in the 3D environment;

a representation of a first participant in the grouped arrangement has a different position than a representation of the first participant in the spatially distributed arrangement; and

a representation of a second participant in the grouped arrangement has a different position than a representation of the second participant in the spatially distributed arrangement.

29. A method, comprising:

at a computer system that is in communication with a display generation component:

while participating in a communication session that is a spatial communication session that includes displaying, via the display generation component, representations of a plurality of participants in the communication session in a spatially distributed arrangement in a 3D environment, wherein displaying the plurality of participants in the spatially distributed arrangement includes displaying:

the representations of the plurality of participants spaced apart from each other and a user of the computer system by at least a threshold amount in a first non-vertical direction in the 3D environment; and

the representations of the plurality of participants spaced apart from each other and the user by at least the threshold amount in a second non-vertical direction that is different from the first non-vertical direction;

while displaying the representations of the plurality of participants distributed in the 3D environment, detecting an event; and

in response to detecting the event, transitioning the communication session from the spatial communication session to a non-spatial communication session that includes displaying, via the display generation component, representations of at least a subset of the plurality of participants of the communication session in a grouped arrangement, wherein in the grouped arrangement:

the representations of the plurality of participants are spaced apart from each other by less than the threshold amount in the first non-vertical direction in the 3D environment;

a representation of a first participant in the grouped arrangement has a different position than a representation of the first participant in the spatially distributed arrangement; and

a representation of a second participant in the grouped arrangement has a different position than a representation of the second participant in the spatially distributed arrangement.

30. The non-transitory computer-readable storage medium of claim 28 , wherein:

in the non-spatial communication session:

a representation of a first participant of the plurality of participants is in a first window region, and

a representation of a second participant of the plurality of participants is in a second window region that is different from the first window region; and

in the spatial communication session:

a representation of the first participant of the plurality of participants is not in a window region, and

a representation of the second participant of the plurality of participants is not in a window region.

31. The non-transitory computer-readable storage medium of claim 28 , wherein the event is a request received during the communication session to transition a representation of the user of the computer system from a 3D representation to a 2D representation.

32. The non-transitory computer-readable storage medium of claim 28 , wherein the event is a request received during the communication session to transition the communication session from the spatial communication session to the non-spatial communication session.

33. The non-transitory computer-readable storage medium of claim 28 , wherein the event is an additional participant joining the communication session.

34. The non-transitory computer-readable storage medium of claim 28 , the one or more programs further including instructions for:

while the communication session is a non-spatial communication session, shifting a position of a respective window region corresponding to a respective participant based on the respective participant moving.

35. The non-transitory computer-readable storage medium of claim 28 , the one or more programs further including instructions for:

while participating in the communication session that is a non-spatial communication session, detecting a second event; and

in response to detecting the second event, transitioning the communication session from the non-spatial communication session to the spatial communication session.

36. The non-transitory computer-readable storage medium of claim 28 , the one or more programs further including instructions for:

while in a spatial communication session, displaying, via the display generation component, a self-view of a representation of the user of the computer system in a self-view window region.

37. The non-transitory computer-readable storage medium of claim 28 , wherein:

during a spatial communication session, a first participant of the communication session is enabled to move a respective representation of the first participant and a second participant of the communication session is enabled to move a respective representation of the second participant; and

during a non-spatial communication session, a user of the computer system is enabled to move respective window regions that include respective representations of the plurality of participants of the communication session.

38. The non-transitory computer-readable storage medium of claim 28 , the one or more programs further including instructions for:

in response to changing between a spatial communication and a non-spatial communication session, displaying, via the display generation component, an indication that a mode of the communication session has changed.

39. The method of claim 29 , wherein:

in the non-spatial communication session:

a representation of a first participant of the plurality of participants is in a first window region, and

a representation of a second participant of the plurality of participants is in a second window region that is different from the first window region; and

in the spatial communication session:

a representation of the first participant of the plurality of participants is not in a window region, and

a representation of the second participant of the plurality of participants is not in a window region.

40. The method of claim 29 , wherein the event is a request received during the communication session to transition a representation of the user of the computer system from a 3D representation to a 2D representation.

41. The method of claim 29 , wherein the event is a request received during the communication session to transition the communication session from the spatial communication session to the non-spatial communication session.

42. The method of claim 29 , wherein the event is an additional participant joining the communication session.

43. The method of claim 29 , further comprising:

while the communication session is a non-spatial communication session, shifting a position of a respective window region corresponding to a respective participant based on the respective participant moving.

44. The method of claim 29 , further comprising:

while participating in the communication session that is a non-spatial communication session, detecting a second event; and

in response to detecting the second event, transitioning the communication session from the non-spatial communication session to the spatial communication session.

45. The method of claim 29 , further comprising:

while in a spatial communication session, displaying, via the display generation component, a self-view of a representation of the user of the computer system in a self-view window region.

46. The method of claim 29 , wherein:

during a spatial communication session, a first participant of the communication session is enabled to move a respective representation of the first participant and a second participant of the communication session is enabled to move a respective representation of the second participant; and

during a non-spatial communication session, a user of the computer system is enabled to move respective window regions that include respective representations of the plurality of participants of the communication session.

47. The method of claim 29 , further comprising:

in response to changing between a spatial communication and a non-spatial communication session, displaying, via the display generation component, an indication that a mode of the communication session has changed.

Assignments (1)

Continuity (3)

References Cited (400)