Granted Patent B2

US 12,620,155 · App. 18/242,363 · Granted May 5, 2026

Inventors: Jesse Chand (San Francisco, CA); Shih-Sang Chiu (Menlo Park, CA)

Assignee: Apple Inc.

G06T13/40G06T13/205H04L65/403

Abstract

Claims (72)

1 . A computer system configured to communicate with one or more display generation components, the computer system comprising:

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

displaying, via the one or more display generation components, representations of a plurality of participants in a real-time communication session in a spatial arrangement of tiles in a three-dimensional space, including concurrently displaying:

a representation of a first participant of the plurality of participants in a first tile in the spatial arrangement of tiles; and

a representation of a second participant of the plurality of participants in a second tile in the spatial arrangement of tiles, wherein the second participant is different from the first participant and the second tile is different from the first tile;

while displaying the representations of the plurality of participants in the real-time communication session in the three-dimensional space, detecting activity associated with the first participant of the plurality of participants;

in response to detecting the activity associated with the first participant of the plurality of participants, modifying the representation of the first participant based on the activity, including moving the first tile relative to the second tile in the spatial arrangement of tiles;

while displaying the representations of the plurality of participants in the real-time communication session in the three-dimensional space, detecting movement, relative to the first participant, of a device being used by the first participant to participate in the real-time communication session; and

in response to detecting the movement, relative to the first participant, of the device being used by the first participant to participate in the real-time communication session, moving the first tile forwards or backwards relative to the second tile in the spatial arrangement of tiles.

2 . The computer system of claim 1 , wherein modifying the representation of the first participant based on the activity includes moving the representation of the first participant in the first tile forward or backward relative to the representation of the second participant in the second tile.

3 . The computer system of claim 1 , wherein modifying the representation of the first participant based on the activity includes changing a size of the representation of the first participant relative to a size of the representation of the second participant.

4 . The computer system of claim 1 , wherein modifying the representation of the first participant based on the activity includes changing an angle of the representation of the first participant relative to an angle of the representation of the second participant.

5 . The computer system of claim 1 , wherein detecting the activity associated with the first participant includes detecting sounds from the first participant and/or movement of the representation of the first participant.

6 . The computer system of claim 1 , wherein the activity associated with the first participant is based on movement of the first participant relative to a point of reference corresponding to the real-time communication session.

7 . The computer system of claim 6 , wherein the movement of the first participant relative to the point of reference corresponding to the real-time communication session is a change in orientation of a face of the first participant relative to the point of reference corresponding to the real-time communication session and modifying the representation of the first participant includes tilting, based on the change in orientation, the representation of the first participant.

8 . The computer system of claim 1 , wherein the activity associated with the first participant is based on movement of a point of reference corresponding to the real-time communication session relative to the first participant.

9 . The computer system of claim 8 , wherein modifying the representation of the first participant based on the activity includes:

in accordance with the activity associated with the first participant including the point of reference corresponding to the real-time communication session moving closer to the first participant, moving the representation of the first participant forward relative to the representation of the second participant; and

in accordance with the activity associated with the first participant including the point of reference corresponding to the real-time communication session moving farther from the first participant, moving the representation of the first participant backward relative to the representation of the second participant.

10 . The computer system of claim 1 , wherein displaying the representations of the plurality of participants in the real-time communication session in the spatial arrangement of tiles in the three-dimensional space includes displaying representations of a first subset of the plurality of participants in a first region and displaying representations of a second subset of the plurality of participants, different from the first subset of the plurality of participants, in a second region that is different from the first region, and wherein the one or more programs further include instructions for:

detecting activity associated with a respective participant of the plurality of participants; and

in response to detecting the activity associated with the respective participant of the plurality of participants:

in accordance with a determination that a representation of the respective participant is of the first subset of the plurality of participants that is displayed in the first region, modifying the representation of the respective participant based on the activity associated with the respective participant; and

in accordance with a determination that the representation of the respective participant is of the second subset of the plurality of participants that is displayed in the second region, forgoing modifying the representation of the respective participant based on the activity associated with the respective participant.

11 . A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that is in communication with one or more display generation components, the one or more programs including instructions for:

displaying, via the one or more display generation components, representations of a plurality of participants in a real-time communication session in a spatial arrangement of tiles in a three-dimensional space, including concurrently displaying:

a representation of a first participant of the plurality of participants in a first tile in the spatial arrangement of tiles; and

a representation of a second participant of the plurality of participants in a second tile in the spatial arrangement of tiles, wherein the second participant is different from the first participant and the second tile is different from the first tile;

while displaying the representations of the plurality of participants in the real-time communication session in the three-dimensional space, detecting activity associated with the first participant of the plurality of participants;

in response to detecting the activity associated with the first participant of the plurality of participants, modifying the representation of the first participant based on the activity, including moving the first tile relative to the second tile in the spatial arrangement of tiles;

while displaying the representations of the plurality of participants in the real-time communication session in the three-dimensional space, detecting movement, relative to the first participant, of a device being used by the first participant to participate in the real-time communication session; and

in response to detecting the movement, relative to the first participant, of the device being used by the first participant to participate in the real-time communication session, moving the first tile forwards or backwards relative to the second tile in the spatial arrangement of tiles.

12 . The non-transitory computer-readable storage medium of claim 11 , wherein modifying the representation of the first participant based on the activity includes moving the representation of the first participant in the first tile forward or backward relative to the representation of the second participant in the second tile.

13 . The non-transitory computer-readable storage medium of claim 11 , wherein modifying the representation of the first participant based on the activity includes changing a size of the representation of the first participant relative to a size of the representation of the second participant.

14 . The non-transitory computer-readable storage medium of claim 11 , wherein modifying the representation of the first participant based on the activity includes changing an angle of the representation of the first participant relative to an angle of the representation of the second participant.

15 . The non-transitory computer-readable storage medium of claim 11 , wherein detecting the activity associated with the first participant includes detecting sounds from the first participant and/or movement of the representation of the first participant.

16 . The non-transitory computer-readable storage medium of claim 11 , wherein the activity associated with the first participant is based on movement of the first participant relative to a point of reference corresponding to the real-time communication session.

17 . The non-transitory computer-readable storage medium of claim 16 , wherein the movement of the first participant relative to the point of reference corresponding to the real-time communication session is a change in orientation of a face of the first participant relative to the point of reference corresponding to the real-time communication session and modifying the representation of the first participant includes tilting, based on the change in orientation, the representation of the first participant.

18 . The non-transitory computer-readable storage medium of claim 11 , wherein the activity associated with the first participant is based on movement of a point of reference corresponding to the real-time communication session relative to the first participant.

19 . The non-transitory computer-readable storage medium of claim 18 , wherein modifying the representation of the first participant based on the activity includes:

in accordance with the activity associated with the first participant including the point of reference corresponding to the real-time communication session moving closer to the first participant, moving the representation of the first participant forward relative to the representation of the second participant; and

in accordance with the activity associated with the first participant including the point of reference corresponding to the real-time communication session moving farther from the first participant, moving the representation of the first participant backward relative to the representation of the second participant.

20 . The non-transitory computer-readable storage medium of claim 11 , wherein displaying the representations of the plurality of participants in the real-time communication session in the spatial arrangement of tiles in the three-dimensional space includes displaying representations of a first subset of the plurality of participants in a first region and displaying representations of a second subset of the plurality of participants, different from the first subset of the plurality of participants, in a second region that is different from the first region, and wherein the one or more programs further include instructions for:

detecting activity associated with a respective participant of the plurality of participants; and

in response to detecting the activity associated with the respective participant of the plurality of participants:

in accordance with a determination that a representation of the respective participant is of the first subset of the plurality of participants that is displayed in the first region, modifying the representation of the respective participant based on the activity associated with the respective participant; and

in accordance with a determination that the representation of the respective participant is of the second subset of the plurality of participants that is displayed in the second region, forgoing modifying the representation of the respective participant based on the activity associated with the respective participant.

21 . A method, comprising:

at a computer system that is in communication with one or more display generation components:

displaying, via the one or more display generation components, representations of a plurality of participants in a real-time communication session in a spatial arrangement of tiles in a three-dimensional space, including concurrently displaying:

a representation of a first participant of the plurality of participants in a first tile in the spatial arrangement of tiles; and

a representation of a second participant of the plurality of participants in a second tile in the spatial arrangement of tiles, wherein the second participant is different from the first participant and the second tile is different from the first tile;

while displaying the representations of the plurality of participants in the real-time communication session in the three-dimensional space, detecting activity associated with the first participant of the plurality of participants;

in response to detecting the activity associated with the first participant of the plurality of participants, modifying the representation of the first participant based on the activity, including moving the first tile relative to the second tile in the spatial arrangement of tiles;

while displaying the representations of the plurality of participants in the real-time communication session in the three-dimensional space, detecting movement, relative to the first participant, of a device being used by the first participant to participate in the real-time communication session; and

in response to detecting the movement, relative to the first participant, of the device being used by the first participant to participate in the real-time communication session, moving the first tile forwards or backwards relative to the second tile in the spatial arrangement of tiles.

22 . The method of claim 21 , wherein modifying the representation of the first participant based on the activity includes moving the representation of the first participant in the first tile forward or backward relative to the representation of the second participant in the second tile.

23 . The method of claim 21 , wherein modifying the representation of the first participant based on the activity includes changing a size of the representation of the first participant relative to a size of the representation of the second participant.

24 . The method of claim 21 , wherein modifying the representation of the first participant based on the activity includes changing an angle of the representation of the first participant relative to an angle of the representation of the second participant.

25 . The method of claim 21 , wherein detecting the activity associated with the first participant includes detecting sounds from the first participant and/or movement of the representation of the first participant.

26 . The method of claim 21 , wherein the activity associated with the first participant is based on movement of the first participant relative to a point of reference corresponding to the real-time communication session.

27 . The method of claim 26 , wherein the movement of the first participant relative to the point of reference corresponding to the real-time communication session is a change in orientation of a face of the first participant relative to the point of reference corresponding to the real-time communication session and modifying the representation of the first participant includes tilting, based on the change in orientation, the representation of the first participant.

28 . The method of claim 21 , wherein the activity associated with the first participant is based on movement of a point of reference corresponding to the real-time communication session relative to the first participant.

29 . The method of claim 28 , wherein modifying the representation of the first participant based on the activity includes:

in accordance with the activity associated with the first participant including the point of reference corresponding to the real-time communication session moving closer to the first participant, moving the representation of the first participant forward relative to the representation of the second participant; and

in accordance with the activity associated with the first participant including the point of reference corresponding to the real-time communication session moving farther from the first participant, moving the representation of the first participant backward relative to the representation of the second participant.

30 . The method of claim 21 , wherein displaying the representations of the plurality of participants in the real-time communication session in the spatial arrangement of tiles in the three-dimensional space includes displaying representations of a first subset of the plurality of participants in a first region and displaying representations of a second subset of the plurality of participants, different from the first subset of the plurality of participants, in a second region that is different from the first region, the method further comprising:

detecting activity associated with a respective participant of the plurality of participants; and

in response to detecting the activity associated with the respective participant of the plurality of participants:

in accordance with a determination that a representation of the respective participant is of the first subset of the plurality of participants that is displayed in the first region, modifying the representation of the respective participant based on the activity associated with the respective participant; and

in accordance with a determination that the representation of the respective participant is of the second subset of the plurality of participants that is displayed in the second region, forgoing modifying the representation of the respective participant based on the activity associated with the respective participant.

Assignments (1)

Continuity (4)

References Cited (400)