IP Library Granted Patent US 9,269,368
Granted Patent B2
US 9,269,368 · App. 14/069,124 · Granted Feb 23, 2016

Speaker-identification-assisted uplink speech processing systems and methods

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,269,368
App. No.
14/069,124
Granted
Feb 23, 2016
Kind
B2
Abstract

Methods, systems, and apparatuses are described for performing speaker-identification-assisted speech processing in an uplink path of a communication device. In accordance with certain embodiments, a communication device includes speaker identification (SID) logic that is configured to identify the identity of a near-end speaker. Knowledge of the identity of the near-end speaker is then used to improve the performance of one or more uplink speech processing algorithms implemented on the communication device.

Claims (71)

1. A method, comprising:

receiving, by one or more speech signal processing stages in an uplink path of a communication device, speaker identification information that identifies a target speaker; and

processing, by each of the one of the one or more speech signal processing stages, a respective version of a speech signal in a manner that takes into account the identity of the target speaker, wherein the one or more speech signal processing stages are at least partially implemented by one or more processors, and wherein the one or more speech signal processing stages include at least a sequential combination of three or more of:

an acoustic echo cancellation stage, a multi-microphone noise reduction stage, a residual echo suppression stage, a single channel dereverberation stage, a wind noise reduction stage, and an automatic speech recognition stage.

2. The method of claim 1 , wherein processing a respective version of the speech signal by the acoustic echo cancellation stage comprises:

determining that a portion of the respective version of the speech signal does not comprise speech spoken by the target speaker based on the speaker identification information;

determining that a portion of a far-end speech signal comprises speech based on second speaker identification information that identifies a second target speaker; and

updating at least one of one or more parameters of at least one acoustic echo cancellation filter used by the acoustic echo cancellation stage and statistics used to derive the one or more parameters in response to determining that the portion of the respective version of the speech signal does not comprise speech spoken by the target speaker and the portion of the far-end speech signal comprises speech.

3. The method of claim 1 , wherein processing a respective version of the speech signal by the multi-microphone noise reduction stage comprises:

determining a noise component of a reference signal received from a reference microphone by removing one or more speech components associated with the target speaker from the reference signal based on the speaker identification information; and

removing an estimated noise component from a portion of the respective version of the speech signal that is based on the determined noise component of the reference signal.

4. The method of claim 1 , wherein processing a respective version of the speech signal by the residual echo suppression stage comprises:

determining that a portion of the respective version of the speech signal does not comprise speech spoken by the target speaker based on the speaker identification information;

determining that a portion of a far-end speech signal comprises speech based on second speaker identification information that identifies a second target speaker; and

increasing a degree of residual echo suppression applied to the portion of the respective version of the speech signal in response to determining that the portion of the respective version of the speech signal does not comprise speech spoken by the target speaker and the portion of the far-end speech signal comprises speech.

5. The method of claim 1 , wherein processing a respective version of the speech signal by the single-channel noise suppression stage comprises:

determining whether a portion of the respective version of the speech signal comprises noise only based at least in part on the speaker identification information; and

in response to at least determining that the portion of the respective version of the speech signal comprises noise only:

updating statistics of noise components of the respective version of the speech signal; and

performing noise suppression on the portion of the respective version of the speech signal based at least on the updated statistics.

6. The method of claim 1 , wherein processing a respective version of the speech signal by the wind noise reduction stage comprises:

determining whether a portion of the respective version of the speech signal comprises a combination of wind noise and a desired source or a combination of wind noise and a non-desired source based on the speaker identification information;

applying a first level of attenuation to the portion of the respective version of the speech signal in response to determining that the portion of the respective version of the speech signal comprises a combination of wind noise and the desired source; and

applying a second level of attenuation to the portion of the respective version of the speech signal that is greater than the first level in response to determining that the portion of the respective version of the speech signal comprises a combination of wind noise and the non-desired source.

7. The method of claim 1 , wherein processing a respective version of the speech signal by the wind noise reduction stage comprises:

determining that a portion of the respective version of the speech signal includes wind noise based on the speaker identification information, the speech signal being provided via a microphone;

determining whether one or more other speech signals received via one or more other respective microphones include wind noise based on the speaker identification information; and

in response to determining that at least one of the one or more other speech signals does not include wind noise, obtaining a replacement signal for the portion of the respective version of the speech signal based on at least one of the one or more other speech signals.

8. The method of claim 1 , wherein processing a respective version of the speech signal by the single channel dereverberation stage comprises:

obtaining an estimate of reverberation included in a portion of the respective version of the speech signal based at least in part on the speaker identification information; and

suppressing the reverberation based on the obtained estimate.

9. The method of claim 1 , wherein processing a respective version of the speech signal by the automatic speech recognition stage comprises:

adapting a generic acoustic model of speech to the target speaker based on the speaker identification information; and

performing automatic speech recognition based at least on the adapted acoustic model and the respective version of the speech signal.

10. A communication device, comprising:

uplink speech processing logic comprising one or more speech signal processing stages, each of the one or more speech signal processing stages being configured to receive speaker identification information that identifies a target speaker and process a respective version of the speech signal in a manner that takes into account the identity of the target speaker, the one or more speech signal processing stages being at least partially implemented by one or more processors, and the one or more speech signal processing stages including at least a sequential combination of three or more of: an acoustic echo cancellation stage, a multi-microphone noise reduction stage, a residual echo suppression stage, a single channel dereverberation stage, a wind noise reduction stage, and an automatic speech recognition stage.

11. The communication device of claim 10 , wherein the acoustic echo cancellation stage is configured to:

determine that a portion of the respective version of the speech signal does not comprise speech based on the speaker identification information;

determine that a portion of a far-end speech signal comprises speech spoken by the target speaker based on second speaker identification information that identifies a second target speaker; and

update at least one of one or more parameters of at least one acoustic echo cancellation filter used by the acoustic echo cancellation stage and statistics used to derive the one or more parameters in response to a determination that the portion of the respective version of the speech signal does not comprise speech spoken by the target speaker and the portion of the far-end speech signal comprises speech.

12. The communication device of claim 10 , wherein the multi-microphone noise reduction stage is configured to:

determine a noise component of a reference signal received from a reference microphone by removing one or more speech components associated with the target speaker from the reference signal; and

remove an estimated noise component from a portion of the respective version of the speech signal that is based on the determined noise component of the reference signal based on the speaker identification information.

13. The communication device of claim 10 , wherein the residual echo suppression stage is configured to:

determine that a portion of the respective version of the speech signal does not comprise speech spoken by the target speaker based on the speaker identification information;

determine that a portion of a far-end speech signal comprises speech based on second speaker identification information that identifies a second target speaker; and

increase a degree of residual echo suppression applied to the portion of the respective version of the speech signal in response to a determination that the portion of the respective version of the speech signal does not comprise speech spoken by the target speaker and the portion of the far-end speech signal comprises speech.

14. The communication device of claim 10 , wherein the single-channel noise suppression stage is configured to:

determine whether a portion of the respective version of the speech signal comprises noise only based at least in part on the speaker identification information; and

in response to at least a determination that the portion of the respective version of the speech signal comprises noise only:

update statistics of noise components of the respective version of the speech signal; and

perform noise suppression on the portion of the respective version of the speech signal based at least on the updated statistics.

15. The communication device of claim 10 , wherein of the wind noise reduction stage is configured to:

determine whether a portion of the respective version of the speech signal comprises wind noise, a non-desired source, or a desired source based on the speaker identification information; and

attenuate the portion of the respective version of the speech signal in response to a determination that the portion of the respective version of the speech signal comprises wind noise or the non-desired source.

16. The communication device of claim 10 , wherein the wind noise reduction stage is configured to:

determine whether a portion of the respective version of the speech signal comprises wind noise only based at least in part on the speaker identification information; and

in response to at least a determination that the portion of the respective version of the speech signal comprises wind noise only:

update an estimate of the energy level of the wind noise; and

perform wind noise reduction on the portion of the respective version of the speech signal based at least on the updated estimate.

17. The communication device of claim 10 , wherein the single channel dereverberation stage is configured to:

obtain an estimate of reverberation included in a portion of the respective version of the speech signal based at least in part on the speaker identification information; and

suppress the reverberation based on the obtained estimate.

18. The communication device of claim 10 , wherein automatic speech recognition stage is configured to:

adapt a generic acoustic model of speech to the target speaker based on the speaker identification information; and

perform automatic speech recognition based on the adapted acoustic model and the respective version of the speech signal.

19. A non-transitory computer readable storage medium having computer program instructions embodied in said non-transitory computer readable storage medium for enabling one or more processors to process a speech signal, the computer program instructions including instructions executable to perform operations comprising:

receiving, by one or more speech signal processing stages in an uplink path of a communication device, speaker identification information that identifies a target speaker; and

processing, by each of the one of the one or more speech signal processing stages, a respective version of a speech signal in a manner that takes into account the identity of the target speaker, wherein the one or more speech signal processing stages include at least a sequential combination of three or more of: an acoustic echo cancellation stage, a multi-microphone noise reduction stage, a residual echo suppression stage, a a single channel dereverberation stage, a wind noise reduction stage, and an automatic speech recognition stage.

20. The non-transitory computer readable storage medium of claim 19 , wherein the one or more speech signal processing stages further includes:

a speech encoding stage.

Assignments (7)
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENT NUMBER 9,385,856 TO 9,385,756 PREVIOUSLY RECORDED AT REEL: 47349 FRAME: 001. ASSIGNOR(S) HEREBY CONFIRMS THE MERGER. Recorded Mar 22, 2019
From: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
To: AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITED
Reel/Frame 051144/0648 →
CORRECTIVE ASSIGNMENT TO CORRECT THE EFFECTIVE DATE PREVIOUSLY RECORDED ON REEL 047229 FRAME 0408. ASSIGNOR(S) HEREBY CONFIRMS THE THE EFFECTIVE DATE IS 09/05/2018. Recorded Oct 29, 2018
From: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
To: AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITED
Reel/Frame 047349/0001 →
MERGER Recorded Oct 4, 2018
From: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
To: AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITED
Reel/Frame 047229/0408 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS Recorded Feb 3, 2017
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: BROADCOM CORPORATION
Reel/Frame 041712/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 1, 2017
From: BROADCOM CORPORATION
To: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
Reel/Frame 041706/0001 →
PATENT SECURITY AGREEMENT Recorded Feb 11, 2016
From: BROADCOM CORPORATION
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 037806/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 14, 2014
From: CHEN, JUIN-HWEY; THYSSEN, JES; NEMER, ELIAS; BORGSTROM, BENGT J.; PANDEY, ASHUTOSH; ZOPF, ROBERT W.
To: BROADCOM CORPORATION
Reel/Frame 032219/0577 →