IP Library › Granted Patent US 12670672

Granted Patent B2

US 12670672 · App. 18/108,852 · Granted Jun 30, 2026

Devices, methods, and graphical user interfaces for three-dimensional user experience sessions in an extended reality environment

Inventors: Philipp Rockel (San Francisco, CA); Gary I. Butcher (Los Gatos, CA); Dorian D. Dargan (Oakland, CA); Amy E. Dedonato (San Francisco, CA); Charles C. Hoyt (Pacifica, CA); Matan Stauber (San Francisco, CA); Hugo D. Verweij (Portola Valley, CA); Kristi E. Bauerly (Los Altos, CA)

Assignee: Apple Inc.

G06T19/006 A61B5/0816 G06F3/013 G06T13/20 G06V10/60 G06V10/761 H04S7/30

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12670672
App. No.: 18/108,852
Granted: Jun 30, 2026
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

The present disclosure relates to techniques for providing computer-generated user experience sessions in an extended reality environment. In some embodiments, a computer system provides a computer-generated user experience session with particles that move based on breathing characteristics of a user. In some embodiments, a computer system provides a computer-generated user experience session with options selected based on characteristics of an XR environment. In some embodiments, a computer system provides a computer-generated user experience session with a soundscape having randomly selected curated sound components.

Claims (231)

1 . A computer system configured to communicate with a display generation component and one or more sensors, the computer system comprising:

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

displaying, via the display generation component, a user interface for a user experience session in an extended reality environment, including:

while the user experience session is active:

detecting, via the one or more sensors, one or more breathing characteristics of a user of the computer system;

displaying a user interface object having a plurality of particles that move based on the one or more breathing characteristics of the user of the computer system, including:

in accordance with a determination that a first breathing event of the user of the computer system satisfies a first set of criteria, wherein the first set of criteria is satisfied when the one or more breathing characteristics of the user indicate that the user is inhaling, displaying the particles of the user interface object moving in a first manner during the first breathing event of the user of the computer system, including:

displaying a first set of one or more particles having a first distance from a location corresponding to a viewpoint of the user of the computer system and having a first amount of movement towards the location corresponding to the viewpoint of the user while the user inhales; and

displaying a second set of one or more particles different from the first set of one or more particles and having a second distance from the location corresponding to the viewpoint of the user of the computer system, different from the first distance, and having a second amount of movement towards the location corresponding to the viewpoint of the user while the user inhales, wherein the second amount of movement towards the location corresponding to the viewpoint of the user is greater than the first amount of movement towards the location corresponding to the viewpoint of the user; and

in accordance with a determination that the first breathing event of the user of the computer system satisfies a second set of criteria, wherein the second set of criteria is satisfied when the one or more breathing characteristics of the user indicate that the user is exhaling, displaying the particles of the user interface object moving in a second manner different from the first manner during the first breathing event of the user of the computer system, including:

displaying the first set of one or more particles having a third distance from the location corresponding to the viewpoint of the user of the computer system and having a first amount of movement away from the location corresponding to the viewpoint of the user while the user exhales; and

displaying the second set of one or more particles different from the first set of one or more particles and having a fourth distance from the location corresponding to the viewpoint of the user of the computer system, different from the third distance, and having a second amount of movement away from the location corresponding to the viewpoint of the user while the user exhales, wherein the second amount of movement away from the location corresponding to the viewpoint of the user is greater than the first amount of movement away from the location corresponding to the viewpoint of the user; and

displaying the user interface object with an animated effect that is based on a predetermined biometric rhythm that is selected by the user of the computer system, including:

in accordance with a determination that the predetermined biometric rhythm is a first biometric rhythm, animating the user interface object based on a first pattern that corresponds to the first biometric rhythm; and

in accordance with a determination that the predetermined biometric rhythm is a second biometric rhythm different from the first biometric rhythm, animating the user interface object based on a second pattern that corresponds to the second biometric rhythm.

2 . The computer system of claim 1 , wherein the computer system is in communication with an audio generation component, and wherein displaying the user interface for the user experience session includes:

prior to the user experience session being active, concurrently:

displaying, via the display generation component, a representation of the plurality of particles for the user interface object; and

outputting, via the audio generation component, an audio soundscape for the user experience session.

3 . The computer system of claim 1 , wherein displaying the user interface for the user experience session includes:

prior to the user experience session being active:

displaying a start option that is selectable to initiate the user experience session; and

displaying an indication of a duration of the user experience session.

4 . The computer system of claim 1 , wherein displaying the user interface for the user experience session includes:

prior to the user experience session being active:

displaying a set of one or more audio options that are selectable to choose an audio guide from a plurality of audio guides for the user experience session;

detecting an input directed to a first audio option, of the set of one or more audio options, that is selectable to choose an audio guide from the plurality of audio guides for the user experience session; and

in response to detecting the input directed to the first audio option, selecting a first audio guide from the plurality of audio guides for the user experience session.

5 . The computer system of claim 1 , wherein the computer system is in communication with an audio generation component, the one or more programs further including instructions for:

while the user experience session is active:

outputting an audio component that has a perceived spatial location that moves based on the one or more breathing characteristics of the user of the computer system, including:

in accordance with a determination that a second breathing event of the user of the computer system satisfies a third set of criteria, outputting the audio component with a first perceived spatial location relative to the user of the computer system; and

in accordance with a determination that the second breathing event of the user of the computer system satisfies a fourth set of criteria, outputting the audio component with a second perceived spatial location relative to the user of the computer system that is different from the first perceived spatial location.

6 . The computer system of claim 1 , wherein a portion of a physical environment of the user of the computer system is visible prior to the user experience session being active, the one or more programs further including instructions for:

initiating the user experience session, including:

displaying, via the display generation component, a dimming effect that gradually decreases visibility of the physical environment.

7 . The computer system of claim 1 , the one or more programs further including instructions for:

while the user experience session is active:

detecting, via the one or more sensors, one or more focus-based characteristics of the user of the computer system; and

outputting feedback based on the one or more focus-based characteristics of the user of the computer system.

8 . The computer system of claim 1 , wherein movement of the plurality of particles is based on a set of one or more simulated physical parameters.

9 . The computer system of claim 1 , wherein displaying the user interface for the user experience session includes:

while the user experience session is active, detecting, via the one or more sensors, gaze data indicative of a gaze of the user of the computer system;

while displaying the user interface object having the plurality of particles that move based on the one or more breathing characteristics of the user of the computer system, detecting updated gaze data; and

in response to detecting the updated gaze data:

in accordance with a determination that the updated gaze data indicates that the gaze of the user exceeds a gaze departure threshold, pausing the user experience session;

in accordance with a determination that the updated gaze data does not indicate that the gaze of the user exceeds the gaze departure threshold, forgoing pausing the user experience session; and

in accordance with a determination that the updated gaze data indicates that the user has regained focus while the user experience session is paused, continuing the user experience session.

10 . The computer system of claim 1 , wherein:

displaying the user interface for the user experience session includes displaying the user interface with a set of visual characteristics selected randomly or pseudorandomly from a set of available visual characteristics; and

the computer system is in communication with an audio generation component, the one or more programs further including instructions for:

outputting, via the audio generation component, an audio soundscape for the user experience session, wherein the audio soundscape is output concurrently with displaying the user interface for the user experience session and outputting the audio soundscape includes outputting the audio soundscape with a first set of two or more audio components selected randomly or pseudorandomly from a set of available audio components.

11 . The computer system of claim 1 , wherein displaying the user interface for the user experience session includes:

while displaying the user interface object having a first displayed state in which the plurality of particles are displayed having a first amount of spacing for a first portion of the user experience session, detecting a transition from the first portion of the user experience session to a second portion of the user experience session; and

in response to detecting the transition from the first portion of the user experience session to the second portion of the user experience session, displaying the user interface object having a second displayed state different from the first displayed state, wherein the plurality of particles are displayed for the second portion of the user experience session having a second amount of spacing different from the first amount of spacing.

12 . The computer system of claim 1 , wherein displaying the user interface for the user experience session includes:

while displaying the user interface object with the plurality of particles having an arrangement with a first average spacing between particles, detecting termination of the user experience session; and

in response to detecting termination of the user experience session, displaying an animation of the plurality of particles moving to an arrangement with a second average spacing between particles, where the second average spacing is smaller than the first average spacing.

13 . The computer system of claim 1 , wherein displaying the user interface for the user experience session includes:

while the user experience session is active and an environment of the user of the computer system is visually obscured, detecting termination of the user experience session; and

in response to detecting termination of the user experience session, initiating termination of the user experience session and gradually increasing visibility of the environment of the user.

14 . The computer system of claim 1 , wherein displaying the user interface for the user experience session includes:

while the user experience session is active, detecting termination of the user experience session; and

in response to detecting termination of the user experience session, initiating termination of the user experience session and displaying an option that is selectable to continue the user experience session.

15 . The computer system of claim 1 , wherein displaying the user interface for the user experience session includes:

while the user experience session is active, detecting termination of the user experience session; and

in response to detecting termination of the user experience session, initiating termination of the user experience session and displaying a history of data related to one or more previous user experience sessions.

16 . The computer system of claim 1 , wherein displaying the user interface for the user experience session includes causing at least a portion of an environment of the user of the computer system to be visible while the user experience session is active.

17 . The computer system of claim 1 , the one or more programs further including instructions for:

while the user experience session is active and while displaying the user interface object having a first displayed orientation relative to the user of the computer system, receiving data indicating a change in position of the user of the computer system from a first position in an environment to a second position in the environment different from the first position; and

in response to receiving the data indicating a change in position of the user of the computer system, displaying the user interface object having a second displayed orientation relative to the user of the computer system different from the first displayed orientation.

18 . The computer system of claim 1 , wherein the user interface for the user experience session is capable of being displayed at one or more external computer systems associated with other users in an environment of the user experience session.

19 . The computer system of claim 1 , the one or more programs further including instructions for:

while the user experience session is active and while displaying the user interface object having the plurality of particles that move based on the one or more breathing characteristics of the user of the computer system, receiving data indicating a change in pose of a portion of the user of the computer system; and

in response to receiving the data indicating the change in pose of the portion of the user of the computer system, updating display of the plurality of particles, including:

in accordance with a determination that the data indicating the change in pose of the portion of the user includes an indication that the portion of the user intersects a displayed location of a respective particle, modifying a displayed characteristic of the respective particle; and

in accordance with a determination that the data indicating the change in pose of the portion of the user does not include an indication that the portion of the user intersects the displayed location of the respective particle, forgoing modifying the displayed characteristic of the respective particle.

20 . A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that is in communication with a display generation component and one or more sensors, the one or more programs including instructions for:

displaying, via the display generation component, a user interface for a user experience session in an extended reality environment, including:

while the user experience session is active:

detecting, via the one or more sensors, one or more breathing characteristics of a user of the computer system;

displaying a user interface object having a plurality of particles that move based on the one or more breathing characteristics of the user of the computer system, including:

displaying the user interface object with an animated effect that is based on a predetermined biometric rhythm that is selected by the user of the computer system, including:

21 . The non-transitory computer-readable storage medium of claim 20 , wherein the computer system is in communication with an audio generation component, and wherein displaying the user interface for the user experience session includes:

prior to the user experience session being active, concurrently:

displaying, via the display generation component, a representation of the plurality of particles for the user interface object; and

outputting, via the audio generation component, an audio soundscape for the user experience session.

22 . The non-transitory computer-readable storage medium of claim 20 , wherein displaying the user interface for the user experience session includes:

prior to the user experience session being active:

displaying a start option that is selectable to initiate the user experience session; and

displaying an indication of a duration of the user experience session.

23 . The non-transitory computer-readable storage medium of claim 20 , wherein displaying the user interface for the user experience session includes:

prior to the user experience session being active:

displaying a set of one or more audio options that are selectable to choose an audio guide from a plurality of audio guides for the user experience session;

in response to detecting the input directed to the first audio option, selecting a first audio guide from the plurality of audio guides for the user experience session.

24 . The non-transitory computer-readable storage medium of claim 20 , wherein the computer system is in communication with an audio generation component, the one or more programs further including instructions for:

while the user experience session is active:

outputting an audio component that has a perceived spatial location that moves based on the one or more breathing characteristics of the user of the computer system, including:

25 . The non-transitory computer-readable storage medium of claim 20 , wherein a portion of a physical environment of the user of the computer system is visible prior to the user experience session being active, the one or more programs further including instructions for:

initiating the user experience session, including:

displaying, via the display generation component, a dimming effect that gradually decreases visibility of the physical environment.

26 . The non-transitory computer-readable storage medium of claim 20 , the one or more programs further including instructions for:

while the user experience session is active:

detecting, via the one or more sensors, one or more focus-based characteristics of the user of the computer system; and

outputting feedback based on the one or more focus-based characteristics of the user of the computer system.

27 . The non-transitory computer-readable storage medium of claim 20 , wherein movement of the plurality of particles is based on a set of one or more simulated physical parameters.

28 . The non-transitory computer-readable storage medium of claim 20 , wherein displaying the user interface for the user experience session includes:

while the user experience session is active, detecting, via the one or more sensors, gaze data indicative of a gaze of the user of the computer system;

in response to detecting the updated gaze data:

in accordance with a determination that the updated gaze data indicates that the gaze of the user exceeds a gaze departure threshold, pausing the user experience session;

in accordance with a determination that the updated gaze data does not indicate that the gaze of the user exceeds the gaze departure threshold, forgoing pausing the user experience session; and

in accordance with a determination that the updated gaze data indicates that the user has regained focus while the user experience session is paused, continuing the user experience session.

29 . The non-transitory computer-readable storage medium of claim 20 , wherein:

the computer system is in communication with an audio generation component, the one or more programs further including instructions for:

30 . The non-transitory computer-readable storage medium of claim 20 , wherein displaying the user interface for the user experience session includes:

31 . The non-transitory computer-readable storage medium of claim 20 , wherein displaying the user interface for the user experience session includes:

32 . The non-transitory computer-readable storage medium of claim 20 , wherein displaying the user interface for the user experience session includes:

while the user experience session is active and an environment of the user of the computer system is visually obscured, detecting termination of the user experience session; and

in response to detecting termination of the user experience session, initiating termination of the user experience session and gradually increasing visibility of the environment of the user.

33 . The non-transitory computer-readable storage medium of claim 20 , wherein displaying the user interface for the user experience session includes:

while the user experience session is active, detecting termination of the user experience session; and

34 . The non-transitory computer-readable storage medium of claim 20 , wherein displaying the user interface for the user experience session includes:

while the user experience session is active, detecting termination of the user experience session; and

35 . The non-transitory computer-readable storage medium of claim 20 , wherein displaying the user interface for the user experience session includes causing at least a portion of an environment of the user of the computer system to be visible while the user experience session is active.

36 . The non-transitory computer-readable storage medium of claim 20 , the one or more programs further including instructions for:

37 . The non-transitory computer-readable storage medium of claim 20 , wherein the user interface for the user experience session is capable of being displayed at one or more external computer systems associated with other users in an environment of the user experience session.

38 . The non-transitory computer-readable storage medium of claim 20 , the one or more programs further including instructions for:

in response to receiving the data indicating the change in pose of the portion of the user of the computer system, updating display of the plurality of particles, including:

39 . A method, comprising:

at a computer system that is in communication with a display generation component and one or more sensors:

displaying, via the display generation component, a user interface for a user experience session in an extended reality environment, including:

while the user experience session is active:

detecting, via the one or more sensors, one or more breathing characteristics of a user of the computer system;

displaying a user interface object having a plurality of particles that move based on the one or more breathing characteristics of the user of the computer system, including:

displaying the user interface object with an animated effect that is based on a predetermined biometric rhythm that is selected by the user of the computer system, including:

40 . The method of claim 39 , wherein the computer system is in communication with an audio generation component, and wherein displaying the user interface for the user experience session includes:

prior to the user experience session being active, concurrently:

displaying, via the display generation component, a representation of the plurality of particles for the user interface object; and

outputting, via the audio generation component, an audio soundscape for the user experience session.

41 . The method of claim 39 , wherein displaying the user interface for the user experience session includes:

prior to the user experience session being active:

displaying a start option that is selectable to initiate the user experience session; and

displaying an indication of a duration of the user experience session.

42 . The method of claim 39 , wherein displaying the user interface for the user experience session includes:

prior to the user experience session being active:

displaying a set of one or more audio options that are selectable to choose an audio guide from a plurality of audio guides for the user experience session;

in response to detecting the input directed to the first audio option, selecting a first audio guide from the plurality of audio guides for the user experience session.

43 . The method of claim 39 , wherein the computer system is in communication with an audio generation component, the method further comprising:

while the user experience session is active:

outputting an audio component that has a perceived spatial location that moves based on the one or more breathing characteristics of the user of the computer system, including:

44 . The method of claim 39 , wherein a portion of a physical environment of the user of the computer system is visible prior to the user experience session being active, the method further comprising:

initiating the user experience session, including:

displaying, via the display generation component, a dimming effect that gradually decreases visibility of the physical environment.

45 . The method of claim 39 , further comprising:

while the user experience session is active:

detecting, via the one or more sensors, one or more focus-based characteristics of the user of the computer system; and

outputting feedback based on the one or more focus-based characteristics of the user of the computer system.

46 . The method of claim 39 , wherein movement of the plurality of particles is based on a set of one or more simulated physical parameters.

47 . The method of claim 39 , wherein displaying the user interface for the user experience session includes:

while the user experience session is active, detecting, via the one or more sensors, gaze data indicative of a gaze of the user of the computer system;

in response to detecting the updated gaze data:

in accordance with a determination that the updated gaze data indicates that the gaze of the user exceeds a gaze departure threshold, pausing the user experience session;

in accordance with a determination that the updated gaze data does not indicate that the gaze of the user exceeds the gaze departure threshold, forgoing pausing the user experience session; and

in accordance with a determination that the updated gaze data indicates that the user has regained focus while the user experience session is paused, continuing the user experience session.

48 . The method of claim 39 , wherein:

the computer system is in communication with an audio generation component, the method further comprising:

49 . The method of claim 39 , wherein displaying the user interface for the user experience session includes:

50 . The method of claim 39 , wherein displaying the user interface for the user experience session includes:

51 . The method of claim 39 , wherein displaying the user interface for the user experience session includes:

while the user experience session is active and an environment of the user of the computer system is visually obscured, detecting termination of the user experience session; and

in response to detecting termination of the user experience session, initiating termination of the user experience session and gradually increasing visibility of the environment of the user.

52 . The method of claim 39 , wherein displaying the user interface for the user experience session includes:

while the user experience session is active, detecting termination of the user experience session; and

53 . The method of claim 39 , wherein displaying the user interface for the user experience session includes:

while the user experience session is active, detecting termination of the user experience session; and

54 . The method of claim 39 , wherein displaying the user interface for the user experience session includes causing at least a portion of an environment of the user of the computer system to be visible while the user experience session is active.

55 . The method of claim 39 , further comprising:

56 . The method of claim 39 , wherein the user interface for the user experience session is capable of being displayed at one or more external computer systems associated with other users in an environment of the user experience session.

57 . The method of claim 39 , further comprising:

in response to receiving the data indicating the change in pose of the portion of the user of the computer system, updating display of the plurality of particles, including: