Noise suppression using multiple sensors of a communication device
View Patent ↗Techniques are described herein that suppress noise using multiple sensors (e.g., microphones) of a communication device. Noise modeling (e.g., estimation of noise basis vectors and noise weighting vectors) is performed with respect to a noise signal during operation of a communication device to provide a noise model. The noise model includes noise basis vectors and noise coefficients that represent noise provided by audio sources other than a user of the communication device. Speech modeling (e.g., estimation of speech basis vectors and speech weighting) is performed to provide a speech model. The speech model includes speech basis vectors and speech coefficients that represent speech of the user. A noisy speech signal is processed using the noise basis vectors, the noise coefficients, the speech basis vectors, and the speech coefficients to provide a clean speech signal.
1. A method comprising:
estimating noise basis vectors with respect to a noise signal that is received from a first sensor of a communication device that is configured to be distal a mouth of a user during operation of the communication device to provide a noise model that represents noise provided by audio sources other than the user;
estimating speech basis vectors, speech weights that correspond to the speech basis vectors, and noise weights that correspond to the noise basis vectors based on a noisy speech signal that is received from a second sensor of the communication device that is configured to be proximate the mouth of the user during the operation of the communication device and further based on the noise basis vectors using a non-negative matrix factorization technique, the noisy speech signal representing a combination of speech and the noise; and
estimating a clean speech signal based on the speech basis vectors and the speech weights, the clean speech signal representing the speech without the noise.
2. The method of claim 1 , wherein estimating the noise basis vectors comprises:
estimating the noise basis vectors using a non-negative matrix factorization technique.
3. The method of claim 1 , wherein estimating the noise basis vectors comprises:
estimating the noise basis vectors using a clustering technique.
4. The method of claim 1 , wherein estimating the noise basis vectors comprises:
applying a blocking matrix to a plurality of signals that are received from a plurality of respective sensors of the communication device to suppress indications of the speech therein, the plurality of signals including the noise signal and the noisy speech signal.
5. The method of claim 1 , wherein estimating the noise basis vectors comprises:
estimating the noise basis vectors on-line based on current and past samples of the noise signal at each time instance of successive time instances to provide respective estimates of the noise basis vectors;
wherein estimating the speech basis vectors, the speech weights, and the noise weights comprises:
estimating the speech basis vectors, the speech weights, and the noise weights on-line based on current and past samples of the noisy speech signal at each of the successive time instances based on the noise basis vectors to provide respective estimates of the speech basis vectors, respective estimates of the speech weights, and respective estimates of the noise weights; and
wherein estimating the clean speech signal comprises:
estimating successive portions of the clean speech signal that correspond to the respective time instances based on the respective estimates of the speech basis vectors and the respective estimates of the speech weights.
6. The method of claim 5 , wherein estimating the successive portions of the clean speech signal comprises:
estimating current samples of the clean speech signal comprising:
identifying a subset of the speech weights that corresponds to the current samples of the noisy speech signal; and
estimating the clean speech signal based on the subset of the speech weights and the speech basis vectors.
7. The method of claim 1 , wherein estimating the speech basis vectors comprises:
estimating the speech basis vectors off-line to provide respective estimates of the speech basis vectors;
storing the estimates of the speech basis vectors to be used on-line for estimating a subsequent clean speech signal during a subsequent operation of the communication device.
8. A method comprising:
estimating noise basis vectors representing a noise component; and
estimating speech basis vectors representing a clean speech component;
estimating speech weights that correspond to the speech basis vectors and noise weights that correspond to the noise basis vectors based on a noisy speech signal, the noise basis vectors, and the speech basis vectors using a non-negative matrix factorization technique; and
estimating a clean speech signal based on the speech basis vectors and the speech weights, the clean speech signal representing the clean speech component.
9. The method of claim 8 , wherein estimating the noise basis vectors comprises:
performing a speech suppression technique with respect to a plurality of signals to suppress indications of speech therein to provide at least one speech-suppressed noise signal; and
determining the noise component based on the at least one speech-suppressed noise signal.
10. The method of claim 8 , wherein estimating the noise basis vectors comprises:
estimating the noise basis vectors on-line based on current and past samples of a noise signal that includes the noise component with regard to each of the successive time instances to provide the respective estimates of the noise basis vectors;
wherein estimating the speech basis vectors comprises:
estimating the speech basis vectors on-line based on current and past samples of the noisy speech signal at each of the successive time instances to provide the respective estimates of the speech basis vectors;
wherein estimating the speech weights and the noise weights comprises:
estimating the speech weights and the noise weights on-line based on the current and past samples of the noisy speech signal, the respective estimates of the noise basis vectors, and the respective estimates of the speech basis vectors; and
wherein estimating the clean speech signal comprises:
estimating successive portions of the clean speech signal comprising:
identifying a subset of the speech weights that corresponds to the current samples of the noisy speech signal; and
estimating the clean speech signal based on the respective estimates of the speech basis vectors and respective subsets of the speech weights that correspond to respective current samples of the noisy speech signal.
11. The method of claim 8 , wherein estimating the speech basis vectors comprises:
estimating the speech basis vectors off-line to provide respective estimates of the speech basis vectors;
storing the estimates of the speech basis vectors to be used on-line for estimating a subsequent clean speech signal.
12. The method of claim 8 , wherein estimating the noise basis vectors comprises:
calculating amplitude modulation spectra of a noise signal that includes the noise component; and
approximating the amplitude modulation spectra of the noise signal based on the noise basis vectors multiplied by the noise weights; and
wherein estimating the speech basis vectors comprises:
calculating amplitude modulation spectra of the noisy speech signal; and
approximating the amplitude modulation spectra of the noisy speech signal based on a combination of the estimated noise basis vectors and the speech basis vectors multiplied by a combination of the noise weights and the speech weights.
13. The method of claim 8 , wherein estimating the noise basis vectors comprises:
calculating magnitude spectra of a noise signal that includes the noise component; and
approximating the magnitude spectra of the noise signal based on the noise basis vectors multiplied by the noise weights; and
wherein estimating the speech basis vectors comprises:
calculating magnitude spectra of the noisy speech signal; and
approximating the magnitude spectra of the noisy speech signal based on a combination of the estimated noise basis vectors and the speech basis vectors multiplied by a combination of the noise weights and the speech weights.
14. The method of claim 8 , wherein estimating the noise basis vectors comprises:
calculating power spectra of a noise signal that includes the noise component; and
approximating the power spectra of the noise signal based on the noise basis vectors multiplied by the noise weights; and
wherein estimating the speech basis vectors comprises:
calculating power spectra of the noisy speech signal; and
approximating the power spectra of the noisy speech signal based on a combination of the estimated noise basis vectors and the speech basis vectors multiplied by a combination of the noise weights and the speech weights.
15. A method comprising:
estimating noise basis vectors with respect to a noise signal that is part of a noisy speech signal, the noisy speech signal representing a combination of noise and speech, comprising:
applying a blocking matrix to a plurality of signals that are received from a plurality of respective sensors of a communication device to suppress indications of the speech therein to obtain an estimate of the noise signal;
estimating speech basis vectors, speech weights that correspond to the speech basis vectors, and noise weights that correspond to the noise basis vectors based on the noisy speech signal and further based on the noise basis vectors using a non-negative matrix factorization technique; and
estimating a clean speech signal based on the speech basis vectors and the speech weights, the clean speech signal representing the speech without the noise.
16. The method of claim 15 , wherein estimating the noise basis vectors comprises:
estimating the noise basis vectors using a non-negative matrix factorization technique.
17. The method of claim 15 , wherein estimating the noise basis vectors comprises:
estimating the noise basis vectors using a clustering technique.
18. The method of claim 15 , wherein estimating the speech basis vectors comprises:
enhancing indications of the speech in the plurality of signals that are received from the plurality of respective sensors based on a beamforming technique.
19. The method of claim 15 , wherein estimating the noise basis vectors comprises:
estimating the noise basis vectors on-line based on current and past samples of the noise signal at each time instance of successive time instances to provide respective estimates of the noise basis vectors;
wherein estimating the speech basis vectors, the speech weights, and the noise weights comprises:
estimating the speech basis vectors, the speech weights, and the noise weights on-line based on current and past samples of the noisy speech signal at each of the successive time instances to provide respective estimates of the speech basis vectors, respective estimates of the speech weights, and respective estimates of the noise weights;
wherein estimating the clean speech signal comprises:
estimating successive portions of the clean speech signal that correspond to the respective time instances based on the respective estimates of the speech basis vectors, the respective estimates of the noise basis vectors, and the respective estimates of the speech weights; and
wherein estimating the successive portions of the clean speech signal comprises:
estimating current samples of the clean speech signal comprising:
identifying a subset of the speech weights that corresponds to the current samples of the noisy speech signal; and
estimating the clean speech signal based on the speech basis vectors and the subset of the speech weights.
20. The method of claim 15 , wherein estimating the speech basis vectors comprises:
estimating the speech basis vectors off-line to provide respective estimates of the speech basis vectors;
storing the estimates of the speech basis vectors to be used on-line for estimating a subsequent clean speech signal.