IP Library Granted Patent US 11,158,333
Granted Patent B2
US 11,158,333 · App. 16/706,519 · Granted Oct 26, 2021

Multi-stream target-speech detection and channel fusion

Inventors: Francesco Nesta (Aliso Viejo, CA); Saeed Mosayyebpour Kaskari (Irvine, CA)
Assignee: SYNAPTICS INCORPORATED
G10L21/0364G10L15/22G10L25/60G10L25/84H04R1/406H04R3/005H04S3/008H04L65/60H04S2400/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,158,333
App. No.
16/706,519
Granted
Oct 26, 2021
Kind
B2
Abstract

Audio processing systems and methods include an audio sensor array configured to receive a multichannel audio input and generate a corresponding multichannel audio signal and target-speech detection logic and an automatic speech recognition engine or VoIP application. An audio processing device includes a target speech enhancement engine configured to analyze a multichannel audio input signal and generate a plurality of enhanced target streams, a multi-stream target-speech detection generator comprising a plurality of target-speech detector engines each configured to determine a probability of detecting a specific target-speech of interest in the stream, wherein the multi-stream target-speech detection generator is configured to determine a plurality of weights associated with the enhanced target streams, and a fusion subsystem configured to apply the plurality of weights to the enhanced target streams to generate an enhancement output signal.

Claims (27)

1. A system comprising:

a target-speech enhancement engine configured to analyze a multichannel audio input signal and generate a plurality of enhanced target streams using different enhancement separation criteria for each of the plurality of enhanced target streams;

a multi-stream target-speech detector generator comprising a plurality of target-speech detector engines each configured to determine a confidence of quality and/or a presence of a specific target-speech in the plurality of enhanced target streams, wherein the multi-stream target-speech detector generator is configured to determine a plurality of weights associated with the plurality of enhanced target streams; and

a fusion subsystem configured to apply the plurality of weights to the plurality of enhanced target streams to generate a combined enhanced output signal.

2. The system of claim 1 , further comprising an audio sensor array configured to sense human speech and environmental noise and generate a corresponding multichannel audio input signal comprising the sensed human speech and environmental noise.

3. The system of claim 1 , wherein the target-speech enhancement engine comprises a plurality of speech enhancement modules, each speech enhancement module configured to analyze the multichannel audio input signal and output one of the plurality of enhanced target streams.

4. The system of claim 3 , wherein the plurality of speech enhancement modules comprise an adaptive spatial filtering algorithm, a beamforming algorithm, a blind source separation algorithm, a single channel enhancement algorithm, and/or a neural network.

5. The system of claim 1 , wherein at least one of the target-speech detector engines comprises Gaussian Mixture Models, Hidden Markov Models, and/or a neural network.

6. The system of claim 1 , wherein each of the target-speech detector engines is further configured to produce a posterior weight correlated to a confidence that an input audio stream includes the specific target-speech.

7. The system of claim 6 , wherein each of the target-speech detector engines is further configured to produce a higher posterior with clean speech.

8. The system of claim 1 , wherein the enhanced output signal is a weighted sum of the plurality of enhanced target streams.

9. The system of claim 1 , wherein the multi-stream target-speech detector generator is further configured to determine a combined probability of detecting a specific target-speech in the plurality of enhanced target streams, and wherein the target-speech is detected if the combined probability exceeds a detection threshold.

10. The system of claim 9 , further comprising an automatic speech recognition engine or a Voice-over-IP (VoIP) application, and wherein the enhanced output signal is forwarded to the automatic speech recognition engine or the VoIP application if the target-speech is detected.

11. A method comprising:

analyzing, using a target-speech enhancement engine, a multichannel audio input signal and generating a plurality of enhanced target streams using different enhancement separation criteria for each of the plurality of enhanced target streams;

determining a probability of detecting a target-speech in each of the plurality of enhanced target streams using a multi-stream target-speech detector generator;

calculating a weight for each of the plurality of enhanced target streams; and

applying the calculated weights to each of the plurality of enhanced target streams to generate an enhanced output signal.

12. The method of claim 11 , further comprising sensing human speech and environmental noise, using an audio sensor array, and generating a corresponding multichannel audio input signal comprising the sensed human speech and environmental noise.

13. The method of claim 11 , wherein analyzing the multichannel audio input signal comprises applying a plurality of speech enhancement modalities, each speech enhancement modality outputting a separate one of the plurality of enhanced target streams.

14. The method of claim 13 , wherein the plurality of speech enhancement modalities comprises an adaptive spatial filtering algorithm, a beamforming algorithm, a blind source separation algorithm, a single channel enhancement algorithm, and/or a neural network.

15. The method of claim 11 , wherein determining the probability of detecting the target-speech in each of the plurality of enhanced target streams comprises applying Gaussian Mixture Models, Hidden Markov Models, and/or a neural network.

16. The method of claim 11 , wherein determining the probability of detecting the target-speech in each of the plurality of enhanced target streams comprises producing a posterior weight correlated to a confidence that an input stream includes a keyword.

17. The method of claim 16 , further comprising producing a higher posterior with clean speech.

18. The method of claim 11 , wherein the enhanced output signal is a weighted sum of the plurality of enhanced target streams.

19. The method of claim 11 , further comprising determining a combined probability of detecting the target-speech in the plurality of enhanced target streams; and wherein the target-speech is detected if the combined probability exceeds a detection threshold.

20. The method of claim 19 , further comprising performing automatic speech recognition on the enhanced output signal if the target-speech is detected.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE SECOND ASSIGNOR'S NAME PREVIOUSLY RECORDED AT REEL: 051345 FRAME: 0694. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT . Recorded Mar 16, 2022
From: NESTA, FRANCESCO; MOSAYYEBPOUR KASKARI, SAEED
To: SYNAPTICS INCORPORATED
Reel/Frame 059375/0113 →
SECURITY INTEREST Recorded Feb 14, 2020
From: SYNAPTICS INCORPORATED
To: WELLS FARGO BANK, NATIONAL ASSOCIATION
Reel/Frame 051936/0103 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2019
From: NESTA, FRANCESCO; MOSAYYEBPOUR, SAEED
To: SYNAPTICS INCORPORATED
Reel/Frame 051345/0694 →