Granted Patent B2

US 12,219,344 · App. 17/485,052 · Granted Feb 4, 2025

Inventors: Xiaoyuan Tu (Sunnyvale, CA); Alexander Singh Alvarado (San Jose, CA)

Assignee: Apple Inc.

H04S7/304G06F3/012H04R1/32H04R5/02H04S2400/05H04S2420/01

Abstract

Claims (86)

1. A method comprising:

centering, using one or more processors, spatial audio in a three-dimensional (3D) virtual audio space;

obtaining, using the one or more processors, first sensor data from an auxiliary device communicatively coupled to a source device, the source device configured to provide the spatial audio and the auxiliary device configured to playback the spatial audio in the 3D virtual audio space;

obtaining, using the one or more processors, second sensor data from one or more motion sensors of the source device;

determining, using the one or more processors, whether the source device and auxiliary device are in a period of mutual quiescence based on the first and second sensor data;

in accordance with determining that the source device and the auxiliary device are in a period of mutual quiescence, re-centering the spatial audio in the 3D virtual auditory space following the period of mutual quiescence; and

rendering, using the one or more processors, the re-centered spatial audio in the 3D virtual auditory space.

2. The method of claim 1 , wherein the spatial audio is re-centered by zeroing out a correction angle at a rate determined by a size of the correction angle.

3. The method of claim 1 , wherein the mutual quiescence is static mutual quiescence, where both the source device and the auxiliary device are static.

4. The method of claim 1 , wherein the mutual quiescence is correlated mutual quiescence, where the first sensor data and the second sensor data are correlated.

5. The method of claim 1 , wherein the spatial audio is re-centered when the period of mutual quiescence exceeds a threshold time.

6. The method of claim 5 , further comprising:

during the period of mutual quiescence:

accumulating, using a first timer, a mutual quiescence time;

detecting, using the one or more processors, a disturbance based on the first sensor data or the second sensor data;

responsive to the detection, accumulating, using a second timer, a disturbance time;

determining, using the one or more processors, that the disturbance has ended based on the first sensor data or the second sensor data;

subtracting, using the one or more processors, the disturbance time from the mutual quiescence time;

resuming, using the one or more processors, accumulating the mutual quiescence time;

determining, using the one or more processors, whether the mutual quiescence time exceeds the threshold time; and

in accordance with the mutual quiescence time exceeding the threshold time, re-centering the spatial audio in the 3D virtual auditory space.

7. The method of claim 6 , further comprising:

during the disturbance, incrementing the second timer and decrementing the first timer; and

after the disturbance has ended, decrementing the second time and incrementing the first timer.

8. The method of claim 6 , wherein the first sensor data and the second sensor data include acceleration data and rotation rate data, and detecting a disturbance based on the first sensor data or the second sensor data, further comprises:

determining whether a source device maximum, low-pass filtered, acceleration data or an auxiliary device maximum, low-pass filtered, acceleration data meets a first threshold;

determining whether a source device maximum acceleration data or an auxiliary device maximum acceleration data meets a second threshold;

determining whether a source device maximum rotation rate or an auxiliary device maximum rotation rate meets a third threshold;

determining whether a source device maximum average rotation rate or an auxiliary device maximum average rotation rate meets a fourth threshold; and

in accordance with the first, second, third and fourth thresholds being met,

determining that a disturbance is detected.

9. The method of claim 1 , further comprising:

during the period of mutual quiescence:

accumulating, using a first timer, a first mutual quiescence time;

detecting a disturbance based on the first sensor data or the second sensor data;

responsive to the detection, accumulating, using a second timer, a disturbance time;

determining that disturbance time has exceeded a threshold time; and

resetting the first timer and the second timer to zero.

10. The method of claim 6 , further comprising:

during the period of mutual quiescence:

determining that a visual anchor was used to re-center the spatial audio; and

resetting the mutual quiescence time to zero.

11. A system comprising:

one or more processors;

memory storing instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising:

centering spatial audio in a three-dimensional (3D) virtual audio space;

obtaining first sensor data from an auxiliary device communicatively coupled to a source device, the source device configured to provide the spatial audio content and the auxiliary device configured to playback the spatial audio in the 3D virtual audio space;

obtaining second sensor data from one or more motion sensors of the source device;

determining whether the source device and auxiliary device are in a period of mutual quiescence based on the first and second sensor data;

in accordance with determining that the source device and the auxiliary device are in a period of mutual quiescence, re-centering the spatial audio in the 3D virtual auditory space following the period of mutual quiescence; and

rendering the re-centered spatial audio in the 3D virtual auditory space.

12. The system of claim 11 , wherein the spatial audio is re-centered by zeroing out a correction angle at a rate determined by a size of the correction angle.

13. The system of claim 11 , wherein the mutual quiescence is static mutual quiescence, where both the source device and the auxiliary device are static.

14. The system of claim 11 , wherein the mutual quiescence is correlated mutual quiescence, where the first sensor and the second sensor are correlated.

15. The system of claim 11 , wherein the spatial audio is re-centered when the period of mutual quiescence exceeds a threshold time.

16. The system of claim 15 , the operations further comprising:

during the period of mutual quiescence:

accumulating, using a first timer, a mutual quiescence time;

detecting a disturbance based on the first sensor data or the second sensor data;

responsive to the detection, accumulating, using a second timer, a disturbance time;

determining that the disturbance has ended based on the first sensor data or the second sensor data;

subtracting the disturbance time from the mutual quiescence time;

resuming accumulating the mutual quiescence time;

determining whether the mutual quiescence time exceeds the threshold time; and

in accordance with the mutual quiescence time exceeding the threshold time, re-centering the spatial audio in the 3D virtual auditory space.

17. The system of claim 16 , the operations further comprising:

during the disturbance, incrementing the second timer and decrementing the first timer; and

after the disturbance has ended, decrementing the second time and incrementing the first timer.

18. The system of claim 16 , wherein the first sensor data and the second sensor data include acceleration data and rotation rate data, and detecting a disturbance based on the first sensor data or the second sensor data, further comprises:

determining whether a source device maximum, low-pass filtered, acceleration data or an auxiliary device maximum, low-pass filtered, acceleration data meets a first threshold;

determining whether a source device maximum acceleration data or an auxiliary device maximum acceleration data meets a second threshold;

determining whether a source device maximum rotation rate or an auxiliary device maximum rotation rate meets a third threshold;

determining whether a source device maximum average rotation rate or an auxiliary device maximum average rotation rate meets a fourth threshold; and

in accordance with the first, second, third and fourth thresholds being met,

determining that a disturbance is detected.

19. The system of claim 11 , the operations further comprising:

during the period of mutual quiescence:

accumulating, using a first timer, a first mutual quiescence time;

detecting a disturbance based on the first sensor data or the second sensor data;

responsive to the detection, accumulating, using a second timer, a disturbance time;

determining that disturbance time has exceeded a threshold time; and

resetting the first timer and the second timer to zero.

20. The system of claim 16 , the operations further comprising:

during the period of mutual quiescence:

determining that a visual anchor was used to re-center the spatial audio; and

resetting the mutual quiescence time to zero.

Assignments (1)

Continuity (2)

References Cited (56)