IP Library Granted Patent US 8,675,890
Granted Patent B2
US 8,675,890 · App. 12/742,907 · Granted Mar 18, 2014

Speaker localization

Inventors: Gerhard Schmidt (Ulm, DE); Tobias Wolff (Neu-Ulm, DE); Markus Buck (Biberach, DE); Olga González Valbuena (Palencia, ES); Günther Wirsching (Eischstätt, DE)
Assignee: Nuance Communications, Inc.
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,675,890
App. No.
12/742,907
Granted
Mar 18, 2014
Kind
B2
Abstract

The present invention relates to a method for localizing a sound source, in particular, a human speaker, comprising detecting sound generated by the sound source by means of a microphone array comprising more than two microphones and obtaining microphone signals, one for each of the microphones, selecting from the microphone signals a pair of microphone signals for a predetermined frequency range based on the distance of the microphones to each other and estimating the angle of the inci-dence of the sound on the microphone array based on the selected pair of micro-phone signals.

Claims (93)

1. A method for localizing a sound source comprising

detecting sound generated by the sound source using a microphone array comprising more than two microphones and obtaining microphone signals, one for each of the microphones;

obtaining within a processor a test function from the microphone signals;

obtaining a function providing a measure for the angle of the incidence of the sound on the microphone array by a Fourier series based on the test function; and

estimating within the processor the angle of the incidence of the sound on the microphone array from the function providing a measure for the angle of the incidence of the sound on the microphone array.

2. The method according to claim 1 further, comprising

selecting from the microphone signals a pair of microphone signals for a predetermined frequency range based on the distance of the microphones to each other.

3. The method according to claim 2 , further comprising:

digitizing the microphone signals and dividing them into microphone sub-band signals before the step of selecting a pair of microphone signals for a predetermined frequency range; and

wherein the pair of microphone signals is a pair of microphone sub-band signals selected for a sub-band depending on the frequency of the sub-band.

4. The method according to claim 3 , wherein the angle of incidence of the sound generated by the sound source is determined from the test function and a steering vector determined for the microphone array.

5. The method according to claim 3 , wherein the test function is a generalized cross power density spectrum of the selected pair of microphone signals.

6. The method according to claim 3 , further comprising:

filtering one of the selected pair of microphone signals by a first adaptive Finite Impulse Response, FIR, filter comprising first filter coefficients;

filtering the other one of the selected pair of microphone signals by a second adaptive Finite Impulse Response, FIR, filter comprising second filter coefficients; and

wherein the test function is constituted by selected ones of the filter coefficients of either the first or the second FIR adaptive filters.

7. The method according to claim 6 , further comprising

normalizing the filter coefficients of one of the first and second adaptive FIR filters such that the i-th coefficients, i being an integer, are maintained real during the adaptation; and

wherein the test function is constituted by the i-th coefficients of the other one of the first and second adaptive FIR filters.

8. A method for localizing a sound source implemented by a computer executing computer program code stored on a non-transitory computer readable medium, comprising

receiving in a processor at least two microphone signals one generated for each microphone of a microphone array in response to sound generated by the sound source;

filtering one of the microphone signals by a first adaptive filter comprising first filter coefficients;

filtering another one of microphone signals by a second adaptive filter comprising second filter coefficients;

normalizing the filter coefficients of one of the first and second adaptive filters such that the i-th coefficients, i being an integer, are maintained real during the adaptation; and

estimating the angle of the incidence of the sound on the microphone array based on the i-th coefficients of the other one of the first and second adaptive filters within the processor.

9. The method according to claim 8 , further comprising weighting the filter coefficients of one of the first and second adaptive filters during the adaptation by 1-ε, ε being a positive real number less than 1.

10. The method according to claim 8 , further comprising:

defining a measure for the estimation of the angle of incidence of the sound generated by the sound source by means of the test function and evaluating this measure for a predetermined range of values of possible angles of incidence of the sound.

11. A computer program product comprising at least one non-transitory computer readable storage medium having computer-executable instructions for localizing a sound source, the computer-executable instructions comprising:

computer code for receiving microphone signals from a microphone array comprising more than two microphones in response to sound generated by the sound source;

computer code for selecting from the microphone signals a pair of microphone signals for a predetermined frequency range based on the distance of the microphones to each other;

computer code for obtaining a test function from the microphone signals;

computer code for obtaining a function providing a measure for the angle of the incidence of the sound on the microphone array by a Fourier series based on the test function; and

computer code for estimating the angle of the incidence of the sound on the microphone array from the function providing a measure for the angle of the incidence of the sound on the microphone array.

12. A signal processing system, comprising

a microphone array comprising more than two microphones each of which is configured to detect sound generated by a sound source and to obtain a microphone signal corresponding to the detected sound;

a control unit configured to select from the microphone signals a pair of microphone signals for a predetermined frequency range based on the distance of the microphones to each other; and

a localization unit configured to:

obtain a test function from the microphone signals;

obtain a function proving a measure for the angle of the incidence of the sound on the microphone array by a Fourier series based on the test function; and estimate the angle of the incidence of the sound on the microphone array based on the selected pair of microphone signals.

13. The signal processing system according to claim 12 , further comprising

filter banks configured to divide the microphone signals corresponding to the detected sound into microphone sub-band signals; and

wherein the control unit is configured to select from the microphone sub-band signals a pair of microphone sub-band signals and wherein the localization unit is configured to estimate the angle of the incidence of the sound on the microphone array based on the selected pair of microphone sub-band signals.

14. The signal processing system according to claim 12 , wherein the localization unit is configured to determine a test function that depends on the angle of incidence of the sound and to estimate the angle of incidence of the sound generated by the sound source on the basis of the test function.

15. The signal processing system according to claim 14 , wherein the localization unit is configured to determine a generalized cross power density spectrum of the selected pair of microphone signals as the test function.

16. The signal processing system according to claim 14 , further comprising

a first adaptive FIR filter comprising first filter coefficients and configured to filter one of the selected pair of microphone signals;

a second adaptive FIR filter comprising second filter coefficients and configured to filter the other one of the selected pair of microphone signals; and

wherein the test function is constituted by selected ones of the first filter coefficients of the first adaptive filtering means or the second filter coefficients of the second adaptive FIR filtering means.

17. The signal processing system according to claim 16 , further comprising

a normalizing unit configured to normalize the filter coefficients of one of the first and second adaptive FIR filters such that the i-th coefficients, i being an integer, are maintained real during the adaptation; and wherein

the localization unit is configured to estimate the angle of the incidence of the sound on the microphone array based on the i-th coefficients of the other one of the first and second adaptive FIR filters.

18. Signal processing system, the system comprising

at least two microphones each of which is configured to detect sound generated by a sound source and to obtain a microphone signal corresponding to the detected sound;

a first adaptive FIR filter comprising first filter coefficients and configured to filter one of the microphone signals;

a second adaptive FIR filter comprising second filter coefficients and configured to filter another other one of the microphone signals; and

a normalizing unit configured to normalize the filter coefficients of one of the first and second adaptive FIR filters such that the i-th coefficients, i being an integer, are maintained real during the adaptation; and

a localization unit configured to estimate the angle of the incidence of the sound on the microphone array based on the i-th coefficients of the other one of the first and second adaptive FIR filters.

19. A video conference system, comprising:

at least one video camera and the signal processing system according to claim 12 and a control unit configured to point the at least one video camera to a direction determined from the estimated angle of incidence of the sound generated by the sound source.

20. A computer program product comprising at least one non-transitory computer readable storage medium having computer-executable instructions for localizing a sound source, the computer-executable instructions comprising:

computer code for receiving microphone signals from a microphone array comprising more than two microphones in response to sound generated by the sound source;

computer code for detecting sound generated by the sound source by a microphone array comprising a first and a second microphone and obtaining microphone signals, one for each of the first and the second microphones;

computer code for obtaining a test function from the microphone signals;

computer code for obtaining a function providing a measure for the angle of the incidence of the sound on the microphone array by a Fourier series based on the test function; and

computer code for estimating the angle of the incidence of the sound on the microphone array from the function providing a measure for the angle of the incidence of the sound on the microphone array.

21. The computer program product according to claim 20 , comprising

computer code for digitizing the microphone signals and dividing them into microphone sub-band signals before the step of selecting a pair of microphone signals for a predetermined frequency range; and

wherein the pair of microphone signals is a pair of microphone sub-band signals selected for a sub-band depending on the frequency of the sub-band.

22. The computer program product according to claim 21 , wherein the computer code for estimating the angle of incidence of the sound generated by the sound source comprises determining a test function that depends on the angle of incidence of the sound.

23. The computer program product according to claim 22 , wherein the angle of incidence of the sound generated by the sound source is determined from the test function and a steering vector determined for the microphone array.

24. The computer program product according to claim 22 , wherein the test function is a generalized cross power density spectrum of the selected pair of microphone signals.

25. The computer program product according to claim 22 , further comprising:

computer code for filtering one of the selected pair of microphone signals by a first adaptive Finite Impulse Response, FIR, filter comprising first filter coefficients;

computer code for filtering the other one of the selected pair of microphone signals by a second adaptive Finite Impulse Response, FIR, filter comprising second filter coefficients; and

wherein the test function is constituted by selected ones of the filter coefficients of either the first or the second FIR adaptive filters.

26. The computer program product according to claim 25 , further comprising

computer code for normalizing the filter coefficients of one of the first and second adaptive FIR filters such that the i-th coefficients, i being an integer, are maintained real during the adaptation; and

wherein the test function is constituted by the i-th coefficients of the other one of the first and second adaptive FIR filters.

27. The computer program product according to claim 11 further comprising

computer code for selecting from the microphone signals a pair of microphone signals for a predetermined frequency range based on the distance of the microphones to each other.

28. The computer program product according to claim 27 , further comprising:

computer code for digitizing the microphone signals and dividing them into microphone sub-band signals before the step of selecting a pair of microphone signals for a predetermined frequency range; and

wherein the pair of microphone signals is a pair of microphone sub-band signals selected for a sub-band depending on the frequency of the sub-band.

29. The computer program product according to claim 28 , wherein computer code for estimating the angle of incidence of the sound generated by the sound source is determined from the test function and a steering vector determined for the microphone array.

30. The computer program product according to claim 27 , wherein the test function is a generalized cross power density spectrum of the selected pair of microphone signals.

31. The computer program product according to claim 27 , further comprising:

computer code for filtering one of the selected pair of microphone signals by a first adaptive Finite Impulse Response, FIR, filter comprising first filter coefficients;

computer code for filtering the other one of the selected pair of microphone signals by a second adaptive Finite Impulse Response, FIR, filter comprising second filter coefficients; and

wherein the test function is constituted by selected ones of the filter coefficients of either the first or the second FIR adaptive filters.

32. The method according to claim 31 , further comprising

computer code for normalizing the filter coefficients of one of the first and second adaptive FIR filters such that the i-th coefficients, i being an integer, are maintained real during the adaptation; and

wherein the test function is constituted by the i-th coefficients of the other one of the first and second adaptive FIR filters.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 1, 2010
From: SCHMIDT, GERHARD; WOLFF, TOBIAS; BUCK, MARKUS; GONZALEZ VALBUENA, OLGA; WIRSCHING, GUNTHER
To: NUANCE COMMUNICATIONS, INC.
Reel/Frame 025079/0636 →
Priority Claims (1)
EP 07022602 · Nov 21, 2007 · regional
Continuity (1)
Related Publication 20110019835A1 · Jan 27, 2011