IP Library Granted Patent US 12,243,514
Granted Patent B2
US 12,243,514 · App. 17/579,766 · Granted Mar 4, 2025

Data augmentation system and method for multi-microphone systems

Inventors: Dushyant Sharma (Mountain House, CA); Ljubomir Milanovic (Vienna, AT); Philipp Salletmayr (Austria, AT); Rong Gong (Vienna, AT); Patrick A. Naylor (Reading, GB)
Assignee: Microsoft Technology Licensing, LLC
G10L15/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,243,514
App. No.
17/579,766
Granted
Mar 4, 2025
Kind
B2
Abstract

A method, computer program product, and computing system for obtaining one or more speech signals from a first device, thus defining one or more first device speech signals. One or more speech signals may be obtained from a second device, thus defining one or more second device speech signals. A noise component model may be selected from a plurality of noise component models based upon, at least in part, the one or more first device speech signals and the one or more second device speech signals. The one or more second device speech signals may be augmented, at run-time, based upon, at least in part, the noise component model.

Claims (54)

1. A computer-implemented method, executed on a computing device, comprising:

obtaining one or more speech signals from a first device, thus defining one or more first device speech signals;

obtaining one or more speech signals from a second device, thus defining one or more second device speech signals;

selecting a noise component model from a plurality of noise component models based upon, at least in part, the one or more first device speech signals and the one or more second device speech signals, wherein the noise component model maps noise spectrum of the first device speech signals to the second device speech signals; and

augmenting, at run-time, the one or more second device speech signals based upon, at least in part, the noise component model.

2. The computer-implemented method of claim 1 , further comprising:

processing, at run-time, the one or more first device speech signals; and

processing, at run-time, the one or more second device speech signals.

3. The computer-implemented method of claim 2 , wherein processing the one or more first device speech signals includes:

detecting one or more speech active portions from the one or more first device speech signals; and

identifying one or more noise components within the one or more first device speech signals.

4. The computer-implemented method of claim 2 , wherein processing the one or more second device speech signals includes:

detecting one or more speech active portions from the one or more second device speech signals; and

identifying one or more noise components within the one or more second device speech signals.

5. The computer-implemented method of claim 1 , wherein selecting a noise component model from a plurality of noise component models based upon, at least in part, the one or more first device speech signals and the one or more second device speech signals includes one or more of:

selecting the noise component model based upon, at least in part, speaker location information associated with at least one of the one or more first device speech signals and the one or more second device speech signals; and

selecting the noise component model based upon, at least in part, an acoustic relative transfer function from a plurality of acoustic relative transfer functions.

6. The computer-implemented method of claim 5 , wherein augmenting, at run-time, the one or more second device speech signals based upon, at least in part, the noise component model includes performing de-reverberation on the one or more second device speech signals based upon, at least in part, the acoustic relative transfer function.

7. The computer-implemented method of claim 1 , wherein augmenting, at run-time, the one or more second device speech signals based upon, at least in part, the noise component model includes performing de-noising on the one or more second device speech signals based upon, at least in part, the noise component model.

8. A computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:

obtaining one or more speech signals from a first device, thus defining one or more first device speech signals;

obtaining one or more speech signals from a second device, thus defining one or more second device speech signals;

selecting a noise component model from a plurality of noise component models based upon, at least in part, the one or more first device speech signals and the one or more second device speech signals, wherein the noise component model maps noise spectrum of the first device speech signals to the second device speech signals; and

augmenting, at run-time, the one or more second device speech signals based upon, at least in part, the noise component model.

9. The computer program product of claim 8 , wherein the operations further comprise:

processing, at run-time, the one or more first device speech signals; and

processing, at run-time, the one or more second device speech signals.

10. The computer program product of claim 9 , wherein processing the one or more first device speech signals includes:

detecting one or more speech active portions from the one or more first device speech signals; and

identifying one or more noise components within the one or more first device speech signals.

11. The computer program product of claim 9 , wherein processing the one or more second device speech signals includes:

detecting one or more speech active portions from the one or more second device speech signals; and

identifying one or more noise components within the one or more second device speech signals.

12. The computer program product of claim 8 , wherein selecting a noise component model from a plurality of noise component models based upon, at least in part, the one or more first device speech signals and the one or more second device speech signals includes one or more of:

selecting the noise component model based upon, at least in part, speaker location information associated with at least one of the one or more first device speech signals and the one or more second device speech signals; and

selecting the noise component model based upon, at least in part, an acoustic relative transfer function from a plurality of acoustic relative transfer functions.

13. The computer program product of claim 12 , wherein augmenting, at run-time, the one or more second device speech signals based upon, at least in part, the noise component model includes performing de-reverberation on the one or more second device speech signals based upon, at least in part, the acoustic relative transfer function.

14. The computer program product of claim 8 , wherein augmenting, at run-time, the one or more second device speech signals based upon, at least in part, the noise component model includes performing de-noising on the one or more second device speech signals based upon, at least in part, the noise component model.

15. A computing system comprising:

a memory; and

a processor configured to obtain one or more speech signals from a first device, thus defining one or more first device speech signals, wherein the processor is further configured to obtain one or more speech signals from a second device, thus defining one or more second device speech signals, wherein the processor is further configured to select a noise component model from a plurality of noise component models based upon, at least in part, the one or more first device speech signals and the one or more second device speech signals, wherein the noise component model maps noise spectrum of the first device speech signals to the second device speech signals, and wherein the processor is further configured to augment, at run-time, the one or more second device speech signals based upon, at least in part, the noise component model.

16. The computing system of claim 15 , wherein the processor is further configured to:

process, at run-time, the one or more first device speech signals; and

process, at run-time, the one or more second device speech signals.

17. The computing system of claim 16 , wherein processing the one or more first device speech signals includes:

detecting one or more speech active portions from the one or more first device speech signals; and

identifying one or more noise components within the one or more first device speech signals.

18. The computing system of claim 15 , wherein processing the one or more second device speech signals includes:

detecting one or more speech active portions from the one or more second device speech signals; and

identifying one or more noise components within the one or more second device speech signals.

19. The computing system of claim 15 , wherein selecting a noise component model from a plurality of noise component models based upon, at least in part, the one or more first device speech signals and the one or more second device speech signals includes one or more of:

selecting the noise component model based upon, at least in part, speaker location information associated with at least one of the one or more first device speech signals and the one or more second device speech signals; and

selecting the noise component model based upon, at least in part, an acoustic relative transfer function from a plurality of acoustic relative transfer functions.

20. The computing system of claim 19 , wherein augmenting, at run-time, the one or more second device speech signals based upon, at least in part, the noise component model includes performing de-reverberation on the one or more second device speech signals based upon, at least in part, the acoustic relative transfer function.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 14, 2023
From: NUANCE COMMUNICATIONS, INC.
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 065578/0676 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 20, 2022
From: SHARMA, DUSHYANT; MILANOVIC, LJUBOMIR; SALLETMAYR, PHILIPP; GONG, RONG; NAYLOR, PATRICK A
To: NUANCE COMMUNICATIONS, INC.
Reel/Frame 058707/0131 →
Continuity (1)
Related Publication 20230230580A1 · Jul 20, 2023
References Cited (7)
US 20060222184A1 · Buck et al. · 2006 [cited by applicant]
US 20120148069A1 · Bai · 2012 [cited by examiner]
US 20160021477A1 · Hiipakka · 2016 [cited by examiner]
US 20160064009A1 · Every · 2016 [cited by examiner]
US 20180054683A1 · Pedersen · 2018 [cited by examiner]
WO WO2020064089A1 · 2020 [cited by examiner]
“International Search Report and Written Opinion Issued in PCT Application No. PCT/US23/060980”, Mailed Date: May 2, 2023, 8 Pages. [cited by applicant]