IP Library Granted Patent US 12,170,867
Granted Patent B2
US 12,170,867 · App. 17/745,214 · Granted Dec 17, 2024

Earbud location detection based on acoustical signature with user-specific customization

Inventor: Gilad Pundak (Rehovot, IL)
Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
H04R1/1041G10L25/51G10L25/78H04R1/08H04R1/1016H04R1/1075
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,170,867
App. No.
17/745,214
Granted
Dec 17, 2024
Kind
B2
Abstract

An earbud is configured to detect its location (e.g., in-ear and out-of-ear) based on an acoustical signature with and without user-specific customization. The earbud location may be indicated to a host, e.g., to determine playback. Location determinations are based on features extracted from acoustical samples taken by the earbud compared to features extracted from out-of-ear acoustical samples and non-user-specific and/or user-specific in-ear samples. A non-user-specific machine learning (ML) model trained on features extracted from non-user-specific in-ear and out-of-ear samples may be an initial/default locator. The non-user-specific model may be customized for specific users. A user-specific in-model may be created by training the non-user-specific model on features extracted from user-specific in-ear samples collected when the earbud is located in-ear for a specific user. The user-specific ML model may be selected to classify a location of the earbud for one or more associated hosts.

Claims (76)

1. An earbud, comprising:

a locator configured to:

determine a location of the earbud as one of a plurality of locations comprising in-ear and out-of-ear locations based on a comparison of features extracted from an acoustical sample taken by the earbud to features extracted from in-ear and out-of-ear acoustical samples; and

indicate the determined location in a location signal transmitted to a host device communicatively connected to the earbud.

2. The earbud of claim 1 , wherein the in-ear acoustical samples comprise non-user-specific in-ear acoustical samples for multiple users.

3. The earbud of claim 1 ,

wherein the locator is further configured to:

perform user-specific in-ear acoustical sampling in an ear of a specific user; and

generate user-specific in-ear acoustical samples based on the user-specific in-ear acoustical sampling; and

wherein the earbud further comprises:

a signal generator to generate a test signal for the in-ear acoustical sampling;

a speaker configured to generate a sound wave from the test signal;

a feedback microphone configured to detect an echo waveform based on the sound wave in the ear of the specific user; and

a signal processor configured to process the echo waveform to generate a user-specific in-ear acoustical sample in the user-specific in-ear acoustical samples.

4. The earbud of claim 3 , further comprising:

a signal combiner configured to combine the test signal with an audio stream of music or an audio stream of a phone call received from a host device to generate a combined signal for output; and

wherein the speaker is configured to generate a sound wave from the combined signal.

5. The earbud of claim 1 , further comprising:

a memory storing at least one machine learning (ML) model configured, upon execution, to perform the determination of the location of the earbud;

wherein the locator is further configured to:

detect that the earbud is connected to a host device;

determine whether the earbud has an ML model associated with the host device;

select a user-specific ML model to perform the determination of the location of the earbud if the earbud is determined to have the user-specific ML model associated with the host device; and

select a non-user-specific ML model to perform the determination of the location of the earbud if the earbud is determined to not have the user-specific ML model associated with the host device.

6. The earbud of claim 5 ,

wherein the locator is further configured to:

detect that the ear-bud is in the ear of a user based on the non-user-specific model;

perform in-ear user-specific learning to generate a user-specific acoustic profile while using the non-user-specific ML model to perform the determination of the location of the earbud; and

generate the user-specific ML model based on the user-specific acoustic profile generated by the in-ear user specific learning.

7. The earbud of claim 5 ,

wherein the locator is further configured to:

use the user-specific model while the location of the earbud is determined to be out-of-ear and while the location of the earbud is determined to be in-ear based on expected acoustical samples for the user-specific model; and

switch from the user-specific model to the non-user-specific model based on an unexpected acoustical sample while the location of the earbud is determined to be in-ear.

8. The earbud of claim 5 , further comprising:

an ML trainer configured to:

extract features from the user-specific in-ear acoustical samples, and

train the non-user-specific ML model based on the extracted features to generate a user-specific ML model.

9. A method performed by an earbud, comprising:

generating an acoustical sample;

extracting features from the acoustical sample;

comparing the features extracted from the acoustical sample to features extracted from in-ear and out-of-ear acoustical samples; and

classifying a location of the earbud as one of a plurality of locations comprising in-ear and out-of-ear locations based on the comparison.

10. The method of claim 9 , further comprising:

transmitting the classified location to a host device communicatively coupled to the earbud.

11. The method of claim 9 , wherein the in-ear acoustical samples comprise non-user-specific in-ear acoustical samples for multiple users.

12. The method of claim 11 , wherein the in-ear acoustical samples also comprise user-specific in-ear acoustical samples in an ear of a specific user.

13. The method of claim 12 , further comprising:

performing, by the earbud, user-specific in-ear acoustical sampling to add the user-specific in-ear acoustical samples to the non-user-specific in-ear acoustical samples.

14. The method of claim 13 , wherein performing the user-specific in-ear acoustical sampling comprises:

performing the user-specific in-ear acoustical sampling during an audio stream of music output through a speaker in the earbud;

performing the user-specific in-ear acoustical sampling during an audio stream of a phone call output through the speaker in the earbud and during voice detection by a microphone in the earbud; and

performing the user-specific in-ear acoustical sampling without an audible audio stream.

15. The method of claim 14 , further comprising:

generating the user-specific in-ear samples by:

emitting an inaudible acoustical waveform from the speaker in the earbud;

detecting an inaudible echo waveform using a feedback microphone in the earbud; and

processing the inaudible echo waveform into the user-specific in-ear samples.

16. The method of claim 11 , further comprising:

detecting that the earbud is connected to a host device;

determining whether the earbud has a machine learning (ML) model associated with the host device;

performing the classifying with the user-specific ML model if the earbud is determined to have the user-specific ML model associated with the host device; and

performing the classifying with a non-user-specific ML model if the earbud is determined to not have the user-specific ML model associated with the host device.

17. The method of claim 16 , further comprising:

detecting that the earbud is in the ear of a user based on the non-user-specific model;

performing in-ear user-specific learning to generate a user-specific acoustic profile while using the non-user-specific ML model to perform the classifying; and

generating the user-specific ML model based on the user-specific acoustic profile generated by the in-ear user specific learning.

18. The method of claim 16 , further comprising:

using the user-specific model while the location of the earbud is classified as out-of-ear and while the location of the earbud is classified as in-ear based on expected acoustical samples for the user-specific model; and

switch from the user-specific model to the non-user-specific model based on an unexpected acoustical sample while the location of the earbud is classified as in-ear.

19. A non-transitory computer-readable storage medium having program instructions recorded thereon that, when executed by a processing circuit, perform a method comprising:

selecting a non-user-specific machine learning (ML) model in the earbud to classify a location of the earbud as one of a plurality of locations comprising in-ear and out-of-ear locations based on features extracted from an acoustical sample taken by the earbud, wherein the non-user-specific ML model is trained on features extracted from non-user-specific in-ear and out-of-ear acoustical samples.

20. The non-transitory computer-readable storage medium of claim 19 , the method further comprising:

detecting that the earbud is in the ear of a user based on the non-user-specific model;

performing in-ear user-specific learning to generate user-specific in-ear samples; and

training the non-user-specific ML model based on features extracted from the user-specific in-ear samples to generate a user-specific ML model; and

selecting the user-specific ML model in the earbud to classify a location of the earbud as one of a plurality of locations comprising in-ear and out-of-ear locations.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 16, 2022
From: PUNDAK, GILAD
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 059919/0679 →
Continuity (1)
Related Publication 20230370760A1 · Nov 16, 2023