IP Library Granted Patent US 11,582,573
Granted Patent B2
US 11,582,573 · App. 17/484,653 · Granted Feb 14, 2023

Disabling/re-enabling head tracking for distracted user of spatial audio application

Inventors: Xiaoyuan Tu (Sunnyvale, CA); Umamahesh Srinivas (Milpitas, CA); Seyyedeh Mahsa Mirzargar (Santa Clara, CA); Keyu Chen (San Ramon, CA); Alexander Singh Alvarado (San Jose, CA); Ronald K. Huang (San Jose, CA); Aditya Sarathy (Santa Clara, CA)
Assignee: Apple Inc.
H04S7/304H04B1/385H04R1/1041H04R5/0335H04S1/005H04W4/80H04W52/0254H04B2001/3866H04R2420/07
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,582,573
App. No.
17/484,653
Granted
Feb 14, 2023
Kind
B2
Abstract

Embodiments are disclosed for disabling/re-enabling head tracking for spatial audio applications. In an embodiment, a method comprises: obtaining, using one or more processors of an auxiliary device worn by a user, motion data; tracking, using the one or more processors, the user's head based at least in part on the motion data; determining, using the one or more processors, whether or not the user is walking based at least in part on the motion data; in accordance with determining that the user is walking, determining if a source device configured to deliver spatial audio to the auxiliary device is static for a specified period of time; and in accordance with determining that the user is walking and the source device is static for the specified period of time, disabling the head tracking.

Claims (96)

1. A method comprising:

obtaining, using one or more processors of an auxiliary device worn by a user, motion data;

tracking, using the one or more processors, the user's head based at least in part on the motion data;

determining, using the one or more processors, whether or not the user is walking based at least in part on the motion data;

in accordance with determining that the user is walking, determining if a source device configured to deliver spatial audio to the auxiliary device is static for a specified period of time; and

in accordance with determining that the user is walking and the source device is static for the specified period of time, disabling the tracking of the user's head.

2. The method of claim 1 , further comprising:

receiving, using a wireless transceiver of the auxiliary device, a radio frequency signal from the source device;

determining, using the one or more processors, that a strength of the radio frequency (RF) signal has decreased by a specified amount; and

in accordance with determining that the user is walking, the source device is static for the specified period of time and the strength of the RF signal has decreased by a specified amount, disabling the head tracking.

3. The method of claim 2 , wherein in the strength of the RF signal is indicated by a received signal strength indicator (RSSI) computed from the signal strength of the received RF signal.

4. The method of claim 1 , wherein determining whether or not the user is walking based at least in part on the motion data, further comprises:

converting the motion data into a frequency spectrum;

determining, based on one or more features of the frequency spectrum, whether or not a foot-strike has occurred that is indicative of walking;

determining a roll angle from the motion data in a face reference frame;

determining a pitch angle from the motion data in the face reference frame;

determining whether or not the roll angle and the pitch angle are within expected ranges for walking; and

in accordance with determining that the one or more features of the frequency spectrum indicate a foot-strike and side sway and the roll and pitch angles are within their expected ranges for walking,

determining that the user is walking.

5. The method of claim 3 , wherein determining, based on one or more features of the frequency spectrum, whether not a foot-strike has occurred that is indicative of walking, further comprises:

converting the motion data into a vertical plane and a horizontal plane in an inertial reference frame;

computing a first frequency spectrum of the vertical motion;

computing a second frequency spectrum of the horizontal motion;

determining a peak energy in the first frequency spectrum;

determining a first portion of energy in the first frequency spectrum and a second portion of the energy in the second frequency spectrum; and

determining a presence of a foot-strike at each step of the user and side sway at each step to step based on the peak energy exceeding a specified threshold and a ratio of the first portion of energy and the second portion of energy.

6. The method of claim 4 , further comprising:

determining that the motion data is missing samples; and

interpolating the motion data to generate additional motion data.

7. The method of claim 6 , where the motion data includes acceleration data and rotation rate data, and interpolating the motion data to generate additional motion data, further comprises:

buffering a specified number of motion data samples in a buffer;

identifying a gap in the buffered motion data samples;

determining whether or not the gap is less than a specified amount of time or data rate;

in accordance with the gap being less than the specified amount of time or data rate, interpolating the buffered motion data samples to a higher number of motion data samples;

for acceleration data in the motion data samples, zero padding the acceleration data, generating a frequency spectrum from the zero padded acceleration data, extracting one or more features from the frequency spectrum that indicate a foot-strike or side sway, and determining that the user is walking based at least in part on the extracted one or more features; and

for rotation rate data in the motion data samples, integrating the rotation rate data over the higher number of rotation rate samples to obtain roll and pitch angles, and determining that the user is walking at least in part based on the roll and pitch angles being within an expected range for walking.

8. The method of claim 1 , wherein determining whether or not the user is walking based at least in part on the motion data is computed every X seconds with an Y % overlap window, where X and Y are integers or real numbers greater than zero.

9. The method of claim 1 , wherein N consecutive determinations that the user is walking are made before disabling the auxiliary device, where N is an integer greater than one.

10. The method of claim 1 , further comprising:

obtaining, using the one or more processors, data from another auxiliary device worn by the user, transition data indicating a sit-to-stand transition or stand-to-sit transition; and

in accordance with determining that the user is walking, the source device is static for the specified period of time and the transition data, disabling the head tracking.

11. A method comprising:

disabling, using one or more processors of an auxiliary device worn by a user, head tracking for a spatial audio application, where a source device is presenting at least spatial audio content to the user through the auxiliary device;

detecting, using the one or more processors, a touch input on a display screen of the source device; or

determining, using the one or more processors, that the auxiliary device is static; or

determining, using the one or more processors, that the source device is moving; or

determining, using the one or more processors, that the user is walking back to the source device; and

in accordance with detecting touch input on the display screen of the source device or determining that the auxiliary device is static or that the source device is moving or the user is walking back to the source device, re-enabling, using the one or more processors, head tracking for the spatial audio application.

12. A system comprising:

one or more processors;

memory storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations comprising:

obtaining motion data from an auxiliary device worn by a user;

tracking a user's head based at least in part on the motion data;

determining whether or not the user is walking based at least in part on the motion data;

in accordance with determining that the user is walking, determining if a source device configured to deliver spatial audio to the auxiliary device is static for a specified period of time; and

in accordance with determining that the user is walking and the source device is static for the specified period of time, disabling the tracking of the user's head.

13. The system of claim 12 , further comprising:

receiving, using a wireless transceiver of the auxiliary, a radio frequency signal from the source device;

determining, using the one or more processors, that a strength of the radio frequency (RF) signal has decreased by a specified amount; and

in accordance with determining that the user is walking, the source device is static for the specified period of time and the strength of the RF signal has decreased by a specified amount, disabling the head tracking.

14. The system of claim 13 , wherein in the strength of the RF signal is indicated by a received signal strength indicator (RSSI) computed from the signal strength of the received RF signal.

15. The system of claim 12 , wherein determining whether or not the user is walking based at least in part on the motion data, further comprises:

converting the motion data into a frequency spectrum;

determining, based on one or more features of the frequency spectrum, whether or not a foot-strike has occurred that is indicative of walking;

determining a roll angle from the motion data in a face reference frame;

determining a pitch angle from the motion data in the face reference frame;

determining whether or not the roll angle and the pitch angle are within expected ranges for walking; and

in accordance with determining that the one or more features of the frequency spectrum indicate a foot-strike and side sway and the roll and pitch angles are within their expected ranges for walking,

determining that the user is walking.

16. The system of claim 15 , wherein determining, based on one or more features of the frequency spectrum, whether not a foot-strike has occurred that is indicative of walking, further comprises:

converting the motion data into a vertical plane and a horizontal plane in an inertial reference frame;

computing a first frequency spectrum of the vertical motion;

computing a second frequency spectrum of the horizontal motion;

determining a peak energy in the first frequency spectrum;

determining a first portion of energy in the first frequency spectrum and a second portion of the energy in the second frequency spectrum; and

determining a presence of a foot-strike at each step of the user and side sway at each step to step based on the peak energy exceeding a specified threshold and a ratio of the first portion of energy and the second portion of energy.

17. The system of claim 16 , further comprising:

determining that the motion data is missing samples; and

interpolating the motion data to generate additional motion data.

18. The system of claim 17 , where the motion data includes acceleration data and rotation rate data, and interpolating the motion data to generate additional motion data, further comprises:

buffering a specified number of motion data samples in a buffer;

identifying a gap in the buffered motion data samples;

determining whether or not the gap is less than a specified amount of time or data rate;

in accordance with the gap being less than the specified amount of time or data rate, interpolating the buffered motion data samples to a higher number of motion data samples;

for acceleration data in the motion data samples, zero padding the acceleration data, generating a frequency spectrum from the zero padded acceleration data, extracting one or more features from the frequency spectrum that indicate a foot-strike or side sway, and determining that the user is walking based at least in part on the extracted one or more features; and

for rotation rate data in the motion data samples, integrating the rotation rate data over the higher number of rotation rate samples to obtain roll and pitch angles, and determining that the user is walking at least in part based on the roll and pitch angles being within an expected range for walking.

19. The system of claim 12 , further comprising:

obtaining, using the one or more processors, data from another auxiliary device worn by the user, transition data indicating a sit-to-stand transition or stand-to-sit transition; and

in accordance with determining that the user is walking, the source device is static for the specified period of time and the transition data, disabling the head tracking.

20. A system comprising:

disabling, using one or more processors of an auxiliary device worn by a user, head tracking for a spatial audio application, where a source device is presenting at least spatial audio content to the user through the auxiliary device;

detecting, using the one or more processors, a touch input on a display screen of the source device; or

determining, using the one or more processors, that the auxiliary device is static; or

determining, using the one or more processors, that the source device is moving; or

determining, using the one or more processors, that the user is walking back to the source device; and

in accordance with detecting touch input on the display screen of the source device or determining that the auxiliary device is static or that the source device is moving or the user is walking back to the source device, re-enabling, using the one or more processors, head tracking for the spatial audio application.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 8, 2021
From: TU, XIAOYUAN; SRINIVAS, UMAMAHESH; MIRZARGAR, SEYYEDEH MAHSA; CHEN, KEYU; SINGH ALVARADO, ALEXANDER; HUANG, RONALD K.; SARATHY, ADITYA
To: APPLE INC.
Reel/Frame 058045/0166 →
Continuity (2)
Provisional Application 63083844 · Sep 25, 2020
Related Publication 20220103964A1 · Mar 31, 2022
Cited By (2)
US 12,219,344 US 12,474,365