IP Library Granted Patent US 9,703,524
Granted Patent B2
US 9,703,524 · App. 14/997,320 · Granted Jul 11, 2017

Privacy protection in collective feedforward

Inventors: Gints Klimanis (SunnyVale, CA); Noah Kraft (New York, NY); Matthew J. Jaffe (San Francisco, CA); Richard Fritz Lanman, III (San Francisco, CA); Jeff Baker (Newbury Park, CA)
Assignee: Doppler Labs, Inc.
G06F3/165G06F3/162G10L25/72H04R1/1083H04L67/18H04L67/30H04R2460/07H04W4/02H04W4/025H04W4/043
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,703,524
App. No.
14/997,320
Granted
Jul 11, 2017
Kind
B2
Abstract

Personal audio systems and methods are disclosed. A personal audio system includes a processor to generate a personal audio stream by processing an ambient audio stream in accordance with an active processing parameter set, a circular buffer memory to store a most recent snippet of the ambient audio stream, and an event detector to detect a trigger event. In response to detection of the trigger event, a controller may extract audio feature data from the most recent snippet of the ambient audio stream and transmit the audio feature data and associated metadata to a knowledgebase remote from the personal audio system.

Claims (85)

1. A personal audio system, comprising:

a processor to generate a personal audio stream by processing an ambient audio stream in accordance with an active processing parameter set, wherein the active processing parameter set indicates a type and degree of one or more processes to be performed on the ambient audio stream;

a buffer memory to store a most recent snippet of the ambient audio stream;

an event detector to detect a trigger event, wherein the trigger event includes retrieving a selected processing parameter set from a dataset memory for use as the active processing parameter set and receiving, via a user interface, a command to modify the active processing parameter set; and

a controller configured to, in response to detection of the trigger event:

extract audio feature data from the most recent snippet of the ambient audio stream; and

transmit the audio feature data and associated metadata to a device remote from the personal audio system, wherein the associated metadata includes an identifier of the selected processing parameter set and the modified processing parameter set associated with the active processing parameter set.

2. The personal audio system of claim 1 , wherein

the associated metadata includes the active processing parameter set before modification.

3. The personal audio system of claim 1 , wherein

the trigger event includes receiving, via the user interface, a command to store the active processing parameter set in a dataset memory, and

the associated metadata transmitted to the device includes the active processing parameter set.

4. The personal audio system of claim 1 , wherein

the associated metadata transmitted to the device includes a descriptor received from a user via a user interface.

5. The personal audio system of claim 1 , further comprising:

a geo-locator to determine a location of the personal audio system, wherein

the associated metadata transmitted to the device includes data indicating the location of the personal audio system.

6. The personal audio system of claim 1 , wherein

the device is a knowledgebase, and

the controller is further configured to

receive, from the knowledgebase, one or more processing parameter sets derived by the knowledgebase from audio feature data and associated metadata received by the knowledgebase from a plurality of personal audio systems, and

store the one or more processing parameter sets received from the knowledgebase in a parameter memory.

7. The personal audio system of claim 1 , wherein

the audio feature data transmitted to the device includes one or more of arithmetic mean, geometric mean, harmonic mean, centroid, variance, standard deviation, spectral skew, kurtosis, spectral shape, rolloff, spread, flatness, spectral flux, Mel frequency cepstral coefficients, pitch, tonal power ratio, harmonic-to-average power ratio, maximum of autocorrelation function, zero crossing rate, RMS power, peak power, crest factor, amplitude/power envelope and attack/decay rates.

8. The personal audio system of claim 1 , further comprising:

a housing;

a microphone, a preamplifier, and an analog to digital converter to convert ambient sound into the ambient audio stream; and

a digital to analog converter, an amplifier, and a speaker to convert the personal audio stream into personal output sound, wherein

the microphone, the preamplifier, the analog to digital converter, the processor, the digital to analog converter, the amplifier, and the speaker are disposed within the housing.

9. The personal audio system of claim 8 , wherein:

the housing is an earbud housing configured to fit, at least partially, within and be supported by a user's ear.

10. A method for processing ambient sound in a personal audio system, comprising:

generating a personal audio stream by processing an ambient audio stream in accordance with an active processing parameter set, wherein the active processing parameter set indicates a type and degree of one or more processes to be performed on the ambient audio stream;

storing a most recent snippet of the ambient audio stream in a buffer memory;

detecting a trigger event, wherein the trigger event includes retrieving a selected processing parameter set from a dataset memory for use as the active processing parameter set and receiving, via a user interface, a command to modify the active processing parameter set; and

in response to detecting the trigger event:

extracting audio feature data from the most recent snippet of the ambient audio stream; and

transmitting the audio feature data and associated metadata to a device remote from the personal audio system, wherein the associated metadata includes an identifier of the selected processing parameter set and the modified processing parameter set associated with the active processing parameter set.

11. The method of claim 10 , wherein

the associated metadata includes the active processing parameter set before modification.

12. The method of claim 10 , wherein

the trigger event includes receiving, via a user interface, a command to store the active processing parameter set in a dataset memory, and

the associated metadata transmitted to the device includes the active processing parameter set.

13. The method of claim 10 , wherein

the associated metadata transmitted to the device includes a descriptor received from a user via the user interface.

14. The method of claim 10 , further comprising:

determining a geo-location of the personal audio system, wherein

the associated metadata transmitted to the device includes data indicating the geo-location of the personal audio system.

15. The method of claim 10 , wherein

the device is a knowledgebase, and

the method further comprises:

receiving, from the knowledgebase, one or more processing parameter sets derived by the knowledgebase from snippets and associated metadata from a plurality of personal audio systems, and

storing the one or more processing parameter sets received from the knowledgebase in a parameter memory.

16. The method of claim 10 , wherein extracting audio feature data from the most recent snippet of the ambient audio stream further comprises:

performing a preliminary analysis of the most recent snippet;

selecting relevant feature data based on results of the preliminary analysis; and

extracting the relevant feature data from the most recent snippet.

17. The method of claim 10 , wherein extracting audio feature data from the most recent snippet of the ambient audio stream further comprises:

requesting, via the user interface, a user to identify a sound captured in the most recent audio snippet;

receiving, via the user interface, a response from the user;

selecting relevant feature data based on the response; and

extracting the relevant feature data from the most recent snippet.

18. A system, comprising:

a knowledgebase; and

a plurality of personal audio systems in communications with the knowledgebase via a network, each personal audio system comprising:

a processor to generate a personal audio stream by processing an ambient audio stream in accordance with an active processing parameter set, wherein the active processing parameter set indicates a type and degree of one or more processes to be performed on the ambient audio stream;

a circular buffer memory to store a most recent snippet of the ambient audio stream;

an event detector to detect a trigger event, wherein the trigger event includes receiving, via a user interface, a command to modify the active processing parameter set; and

a controller configured to, in response to detection of the trigger event:

extract audio feature data from the most recent snippet of the ambient audio stream; and

transmit the audio feature data and associated metadata to a knowledgebase remote from the personal audio system, wherein the associated metadata includes the modified processing parameter set associated with the active processing parameter set,

wherein the knowledgebase is configured to create new and/or revised processing parameter sets based on the audio feature data and metadata collectively received from the plurality of personal audio systems and to transmit the new and/or revised processing parameter sets to at least some of the plurality of personal audio systems.

19. The system of claim 18 , wherein the knowledgebase is configured to push the new and/or revised processing parameter sets to the plurality of personal audio system on an as-available basis.

20. The system of claim 18 , wherein the knowledgebase is configured to transmit the new and/or revised processing parameter sets to a particular personal audio system from the plurality of personal audio systems in response to a request from the particular personal audio system.

21. The system of claim 18 , wherein the knowledgebase is configured to transmit the new and/or revised processing parameter sets to the at least some of the plurality of personal audio systems at predetermined periodic intervals.

22. A method for processing ambient sound in a personal audio system, comprising:

generating a personal audio stream by processing an ambient audio stream in accordance with an active processing parameter set;

storing a most recent snippet of the ambient audio stream in a buffer memory;

detecting a trigger event; and

in response to detecting the trigger event:

determining whether or not the most recent audio snippet contains speech; and

in the event the most recent audio snippet contains speech:

extracting audio feature data from the most recent snippet of the ambient audio stream; and

transmitting the audio feature data and associated metadata to a device remote from the personal audio system; and

in the event the most recent audio snippet does not contain speech, transmitting the most recent audio snippet and associated metadata to the device remote from the personal audio system.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 23, 2018
From: DOPPLER LABS, INC.
To: DOLBY LABORATORIES LICENSING CORPORATION
Reel/Frame 044703/0475 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 2, 2016
From: KLIMANIS, GINTS; KRAFT, NOAH; BAKER, JEFF; LANMAN, RICHARD FRITZ, III; JAFFE, MATTHEW J.
To: DOPPLER LABS, INC.
Reel/Frame 037647/0648 →
Continuity (2)
Continuation In Part 14952761 · Nov 25, 2015
Related Publication 20170147281A1 · May 25, 2017