IP Library Granted Patent US 10,262,672
Granted Patent B2
US 10,262,672 · App. 15/658,842 · Granted Apr 16, 2019

Audio processing for speech

Inventors: Youhong Lu (Irvine, CA); Ravi Kalluri (San Jose, CA); Andrew Walters (Castroville, CA); Luigi Bojan (Orlando, FL)
Assignee: Verizon Patent and Licensing Inc.
G10L21/0205G10L21/0224G10L21/0232G10L25/21G10L2021/02082
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,262,672
App. No.
15/658,842
Granted
Apr 16, 2019
Kind
B2
Abstract

A method, a device, and a non-transitory storage medium are described in which a power of late reverberation of a speech signal is estimated based on early samples of the speech signal. The power of the late reverberation may be subtracted linearly or non-linearly from the speech signal.

Claims (71)

1. A method comprising:

receiving, by a microphone of a device and from a user, a speech signal, wherein the speech signal includes an early reverberation portion and a late reverberation portion, and wherein the early reverberation portion includes a direct coupling portion in which a path of the user's speech to the device travels a shortest distance to the device and an early reflection portion;

transforming, by a filter of the device, the speech signal to a time and frequency domain signal;

filtering, by the filter of the device, the time and frequency domain signal based on a frequency band included in the time and frequency domain signal;

estimating, by a de-reverber of the device, a power of the late reverberation portion based on a filtered early reverberation portion and a segment of the speech signal in which the user's speech has stopped;

subtracting, by the de-reverber, the estimated power of the late reverberation portion from the speech signal based on the estimating; and

outputting, by the de-reverber, a resultant speech signal based on the subtracting.

2. The method of claim 1 , wherein the subtracting is performed linearly.

3. The method of claim 1 , wherein the subtracting is performed non-linearly.

4. The method of claim 1 , wherein the estimating further comprises:

selecting, by the de-reverber of the device, the segment of the speech signal in which the user's speech has stopped;

calculating, by the de-reverber of the device, a first power of a first point in the segment;

calculating, by the de-reverber of the device, a second power of a second point in the segment that occurs subsequent to the first point; and

estimating, by the de-reverber of the device, an adaptive filter based on the powers of the first point and the second point.

5. The method of claim 4 , further comprising:

estimating, by the de-reverber of the device, the power of the late reverberation portion based on the estimated adaptive filter and a power of the early reverberation portion.

6. The method of claim 1 , wherein the segment has an envelope of power of a room impulse response of a room in which the user and the device reside.

7. The method of claim 1 , further comprising:

selecting, by the de-reverber of the device, the segment of the speech signal in which the user's speech has stopped; and

calculating, by the de-reverber of the device, a first power of the segment.

8. The method of claim 7 , further comprising:

calculating, by the de-reverber of the device, a second power of the segment that occurs after early reverberation;

estimating, by the de-reverber of the device, an adaptive filter based on a ratio of the second power to the first power; and

estimating, by the de-reverber of the device, the power of the late reverberation portion based on the estimated adaptive filter and a power of the early reverberation portion.

9. A device comprising:

a microphone;

an audio processing system including a filter and a de-reverber;

a memory, wherein the memory stores instructions of the audio processing system; and

a processor;

receive, via the microphone and from a user, a speech signal, wherein the speech signal includes an early reverberation portion and a late reverberation portion, and wherein the early reverberation portion includes a direct coupling portion in which a path of the user's speech to the device travels a shortest distance to the device and an early reflection portion; and

wherein the processor executes the instructions to:

transform, by the filter, the speech signal to a time and frequency domain signal;

filter, by the filter, the time and frequency domain signal based on a frequency band included in the time and frequency domain signal;

estimate, by the de-reverber, a power of the late reverberation portion based on a filtered early reverberation portion and a segment of the speech signal in which the user's speech has stopped;

subtract, by the de-reverber, the estimated power of the late reverberation portion from the speech signal based on the estimation; and

output, by the de-reverber, a resultant speech signal based on the subtraction.

10. The device of claim 9 , wherein the subtraction is performed non-linearly or linearly.

11. The device of claim 9 , wherein the processor further executes the instructions to:

select, by the de-reverber, the segment of the speech signal in which the user's speech has stopped;

select, by the de-reverber, another segment of the speech signal that occurs subsequent to the segment; and

estimate, by the de-reverber, an adaptive filter based on the powers of the segment and the other segment.

12. The device of claim 11 , wherein the other segment of the speech signal ends at a silence level or a noise level.

13. The device of claim 11 , wherein the processor further executes the instructions to:

estimate, by the de-reverber, the power of the late reverberation portion based on the estimated adaptive filter and a power of the early reverberation portion.

14. The device of claim 9 , wherein the processor further executes the instructions to:

select, by the de-reverber of the device, the segment of the speech signal in which the user's speech has stopped;

calculate, by the de-reverber of the device, a first power of a first point in the segment;

calculate, by the de-reverber of the device, a second power of a second point in the segment that occurs subsequent to the first point; and

estimate, by the de-reverber of the device, an adaptive filter based on the powers of the first point and the second point.

15. The device of claim 14 , wherein the processor further executes the instructions to:

estimate, by the de-reverber, the power of the late reverberation portion based on the estimated adaptive filter and a power of the early reverberation portion.

16. A non-transitory, computer-readable storage medium storing instructions executable by a processor of a computational device, which when executed cause the computational device to:

receive, from a user via a microphone, a speech signal, wherein the speech signal includes an early reverberation portion and a late reverberation portion, and wherein the early reverberation portion includes a direct coupling portion in which a path of the user's speech to the computational device travels a shortest distance to the computational device and an early reflection portion;

transform the speech signal to a time and frequency domain signal;

filter the time and frequency domain signal based on a frequency band included in the time and frequency domain signal;

estimate a power of the late reverberation portion based on a filtered early reverberation portion and a segment of the speech signal in which the user's speech has stopped;

subtract the estimated power of the late reverberation portion from the speech signal based on the estimation; and

output a resultant speech signal based on the subtraction.

17. The non-transitory, computer-readable storage medium of claim 16 , wherein the instructions further include instructions executable by the processor of the computational device, which when executed cause the computational device to:

select the segment of the speech signal in which the user's speech has stopped;

select another segment of the speech signal that occurs subsequent to the segment;

estimate an adaptive filter based on the powers of the segment and the other segment; and

estimate the power of the late reverberation portion based on the estimated adaptive filter and a power of the early reverberation portion.

18. The non-transitory, computer-readable storage medium of claim 17 , wherein the other segment of the speech signal ends at a silence level or a noise level.

19. The non-transitory, computer-readable storage medium of claim 16 , wherein the instructions further include instructions executable by the processor of the computational device, which when executed cause the computational device to:

select the segment of the speech signal in which the user's speech has stopped;

calculate a first power of a first point in the segment;

calculate a second power of a second point in the segment that occurs subsequent to the first point; and

estimate an adaptive filter based on the powers of the first point and the second point.

20. The non-transitory, computer-readable storage medium of claim 19 , wherein the instructions further include instructions executable by the processor of the computational device, which when executed cause the computational device to:

estimate the power of the late reverberation portion based on the estimated adaptive filter and a power of the early reverberation portion.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 25, 2017
From: LU, YOUHONG; KALLURI, RAVI; WALTERS, ANDREW; BOJAN, LUIGI
To: VERIZON PATENT AND LICENSING INC.
Reel/Frame 043091/0392 →
Continuity (1)
Related Publication 20190035415A1 · Jan 31, 2019