IP Library Granted Patent US 11,837,228
Granted Patent B2
US 11,837,228 · App. 17/314,601 · Granted Dec 5, 2023

System and method for data augmentation for multi-microphone signal processing

Inventors: Dushyant Sharma (Woburn, MA); Patrick A. Naylor (Reading, GB); Rong Gong (Vienna, AT); Stanislav Kruchinin (Vienna, AT); Ljubomir Milanovic (Vienna, AT)
Assignee: Nuance Communications, Inc.
G10L15/22G06F16/65G06F16/686G06N20/00G10L15/20G10L15/32G10L17/06G10L21/028G10L25/78G10L25/84G16H15/00H04R1/406H04R3/005H04R3/04H04R5/04H04R29/005H04S7/307G10L15/26G10L21/0216G10L21/0272G10L2021/02166G16H10/60G16H40/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,837,228
App. No.
17/314,601
Granted
Dec 5, 2023
Kind
B2
Abstract

A method, computer program product, and computing system for receiving a speech signal from each microphone of a plurality of microphones, thus defining a plurality of signals. One or more noise signals associated with microphone self-noise may be received. One or more self-noise-based augmentations may be performed on the plurality of signals based upon, at least in part, the one or more noise signals associated with microphone self-noise, thus defining one or more self-noise-based augmented signals.

Claims (32)

1. A computer-implemented method, executed on a computing device, comprising:

receiving a speech signal from each microphone of a plurality of microphones, thus defining a plurality of signals;

receiving one or more noise signals associated with microphone self-noise;

performing one or more self-noise-based augmentations on the plurality of signals based upon, at least in part, the one or more noise signals associated with microphone self-noise, thus defining one or more self-noise-based augmented signals;

receiving a selection of a target microphone array from a library of target microphone arrays by providing particular self-noise characteristics associated with the target microphone array; and

performing one or more self-noise-based augmentations on the plurality of signals based upon, at least in part, the target microphone array.

2. The computer-implemented method of claim 1 , wherein receiving the one or more noise signals associated with microphone self-noise includes simulating a model representative of microphone self-noise.

3. The computer-implemented method of claim 1 , receiving the one or more noise signals associated with microphone self-noise includes measuring self-noise from at least one microphone.

4. The computer-implemented method of claim 1 , wherein performing the one or more self-noise-based augmentations on the plurality of signals based upon, at least in part, the one or more noise signals associated with microphone self-noise includes adding a noise signal from the one or more noise signals to the speech signal from each microphone.

5. The computer-implemented method of claim 4 , wherein adding a noise signal to the speech signal from each microphone includes adding a noise signal from the one or more noise signals to the speech signal from each microphone based upon, at least in part, a predefined signal-to-noise ratio for the one or more self-noise-based augmented signals.

6. The computer-implemented method of claim 4 , wherein adding a noise signal from the one or more noise signals to the speech signal from each microphone includes adding a random noise signal from the one or more noise signals to the speech signal from each microphone.

7. The computer-implemented method of claim 1 , wherein the plurality of microphones define a microphone array.

8. A computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:

receiving a speech signal from each microphone of a plurality of microphones, thus defining a plurality of signals;

receiving one or more noise signals associated with microphone self-noise;

performing one or more self-noise-based augmentations on the plurality of signals based upon, at least in part, the one or more noise signals associated with microphone self-noise, thus defining one or more self-noise-based augmented signals;

receiving a selection of a target microphone array from a library of target microphone arrays by providing particular self-noise characteristics associated with the target microphone array; and

performing one or more self-noise-based augmentations on the plurality of signals based upon, at least in part, the target microphone array.

9. The computer program product of claim 8 , wherein receiving the one or more noise signals associated with microphone self-noise includes simulating a model representative of microphone self-noise.

10. The computer program product of claim 8 , wherein receiving the one or more noise signals associated with microphone self-noise includes measuring self-noise from at least one microphone.

11. The computer program product of claim 8 , wherein performing the one or more self-noise-based augmentations on the plurality of signals based upon, at least in part, the one or more noise signals associated with microphone self-noise includes adding a noise signal from the one or more noise signals to the speech signal from each microphone.

12. The computer program product of claim 11 , wherein adding a noise signal to the speech signal from each microphone includes adding a noise signal from the one or more noise signals to the speech signal from each microphone based upon, at least in part, a predefined signal-to-noise ratio for the one or more self-noise-based augmented signals.

13. The computer program product of claim 11 , wherein adding a noise signal from the one or more noise signals to the speech signal from each microphone includes adding a random noise signal from the one or more noise signals to the speech signal from each microphone.

14. The computer program product of claim 8 , wherein the plurality of microphones define a microphone array.

15. A computing system comprising:

a memory; and

a processor configured to receive a speech signal from each microphone of a plurality of microphones, thus defining a plurality of signals, wherein the processor is further configured to receive one or more noise signals associated with microphone self-noise, wherein the processor is further configured to perform one or more self-noise-based augmentations on the plurality of signals based upon, at least in part, the one or more noise signals associated with microphone self-noise, thus defining one or more self-noise-based augmented signals, wherein the processor is further configured to receive a selection of a target microphone array from a library of target microphone arrays by providing particular self-noise characteristics associated with the target microphone array, and wherein the processor is further configured to perform one or more self-noise-based augmentations on the plurality of signals based upon, at least in part, the target microphone array.

16. The computing system of claim 15 , wherein receiving the one or more noise signals associated with microphone self-noise includes simulating a model representative of microphone self-noise.

17. The computing system of claim 15 , wherein receiving the one or more noise signals associated with microphone self-noise includes measuring self-noise from at least one microphone.

18. The computing system of claim 15 , wherein performing the one or more self-noise-based augmentations on the plurality of signals based upon, at least in part, the one or more noise signals associated with microphone self-noise includes adding a noise signal from the one or more noise signals to the speech signal from each microphone.

19. The computing system of claim 18 , wherein adding a noise signal to the speech signal from each microphone includes adding a noise signal from the one or more noise signals to the speech signal from each microphone based upon, at least in part, a predefined signal-to-noise ratio for the one or more self-noise-based augmented signals.

20. The computing system of claim 18 , wherein adding a noise signal from the one or more noise signals to the speech signal from each microphone includes adding a random noise signal from the one or more noise signals to the speech signal-based voice data.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 9, 2023
From: NUANCE COMMUNICATIONS, INC.
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 065530/0871 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 7, 2021
From: SHARMA, DUSHYANT; NAYLOR, PATRICK A.; GONG, RONG; KRUCHININ, STANISLAV; MILANOVIC, LJUBOMIR
To: NUANCE COMMUNICATIONS, INC.
Reel/Frame 056172/0372 →