IP Library Granted Patent US 8,606,566
Granted Patent B2
US 8,606,566 · App. 12/126,682 · Granted Dec 10, 2013

Speech enhancement through partial speech reconstruction

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,606,566
App. No.
12/126,682
Granted
Dec 10, 2013
Kind
B2
Abstract

A system improves speech intelligibility by reconstructing speech segments. The system includes a low-frequency reconstruction controller programmed to select a predetermined portion of a time domain signal. The low-frequency reconstruction controller substantially blocks signals above and below the selected predetermined portion. A harmonic generator generates low-frequency harmonics in the time domain that lie within a frequency range controlled by a background noise modeler. A gain controller adjusts the low-frequency harmonics to substantially match the signal strength to the time domain original input signal.

Claims (47)

1. A system that improves speech intelligibility by reconstructing speech segments comprising:

a low-frequency reconstruction controller programmed to select a predetermined portion of a time domain speech signal while substantially blocking or substantially attenuating signals above and below the selected predetermined portion;

a harmonic generator coupled to the low-frequency reconstruction controller programmed to generate low-frequency harmonics of reconstructed speech in the time domain that lie within a frequency range controlled by a background noise modeler;

a gain controller configured to adjust the low-frequency harmonics to substantially match the signal strength in the time domain signal; and

a lowpass filter having a frequency response based on a dynamic noise from changing noise conditions within a vehicle, the lowpass filter configured to receive the adjusted low-frequency harmonics and output a selected portion of the adjusted low-frequency harmonics based on the frequency response and a threshold.

2. The system that improves speech intelligibility of claim 1 where the gain controller comprises a weighting filter programmed to emphasize the low-frequency harmonics during time periods of minimal speech identified by a voice activity detector.

3. The system that improves speech intelligibility of claim 1 where the gain controller comprises a weighting filter programmed to emphasize the low-frequency harmonics when high levels of background noise is detected by a noise detector.

4. The system that improves speech intelligibility of claim 1 where the signal strength comprises a power level.

5. A system that improves speech intelligibility by reconstructing speech comprising:

a first filter that passes a portion of an input signal within a varying range while substantially blocking signals above and below the varying range;

a non-linear transformation controller configured to generate harmonics of reconstructed speech in the time domain;

a multiplier configured to adjust the amplitudes of the harmonics based on an estimated energy in the input signal; and

a second filter in communication with the multiplier having a frequency response based on a dynamic noise from changing noise conditions within a vehicle that is detected in the input signal, the second filter configured to receive the amplitude-adjusted harmonics and select a portion of the amplitude-adjusted harmonics based on the frequency response while minimizing or dampening a remaining portion.

6. The system that improves speech intelligibility of claim 5 where the first filter comprises:

an electronic circuit that passes substantially all frequencies in the input signal that are above a predetermined frequency.

7. The system that improves speech intelligibility of claim 6 where the first filter further comprises:

a second electronic circuit that allows nearly all frequencies in the input signal that are below a predetermined frequency to pass through it.

8. The system that improves speech intelligibility of claim 5 , further comprising:

a spectral converter that is configured to digitize and convert the input signal into the frequency domain;

a background noise estimator configured to measure a background noise that is present in the input signal;

a spectral separator in communication with the spectral converter and the background noise estimator that is configured to divide a power spectrum of a noise estimate; and

a modeler in communication with the spectral separator that fits a plurality of substantially linear functions to differing portions of the background noise estimate;

where the frequency response of the second filter is based on the plurality of substantially linear functions.

9. The system that improves speech quality of claim 8 where the modeler is configured to approximate a plurality of linear relationships.

10. The system that improves speech quality of claim 9 where the modeler is configured to fit a line to a portion of a medium to low frequency portion of an aural spectrum and a line to a high frequency portion of the aural spectrum.

11. The system that improves speech quality of claim 8 where the background noise estimator comprises a power estimator.

12. A system that reconstructs speech in real time comprising:

an input filter that passes a band limited frequency in an aural bandwidth when a speech is detected;

a harmonic generator programmed to reconstruct portions of speech masked by a dynamic noise from changing noise conditions within a vehicle, the harmonic generator generating harmonics of reconstructed speech that occur in a full frequency range of the input filter;

a gain controller that dynamically adjusts the signal strength of the generated harmonics to a targeted level based on a signal within the aural bandwidth;

a speech reconstruction filter that receives the dynamically adjusted harmonics and allows a portion of the dynamically adjusted harmonics to pass through it based on a frequency response of the speech construction filter and a threshold, the frequency response based on the dynamic noise; and

a perceptual filter configured to combine an output of the speech reconstruction filter with the original input speech signal.

13. The system that reconstructs speech in real time of claim 12 where the passband of the input filter occurs near a low frequency range where speech harmonics are likely to be corrupted by noise.

14. The system that reconstructs speech in real time of claim 12 where the adjustment is based on a power ratio between the original input signal and the reconstructed signal.

15. The system that reconstructs speech in real time of claim 12 where the gain controller continuously varies the signal strength of the generated harmonics.

16. The system that reconstructs speech in real time of claim 12 where the harmonic generator is programmed to process a non-linear function.

17. The system that reconstructs speech in real time of claim 12 further comprising means to detect speech.

18. A method that compensates for undesired changes in a speech segment, comprising:

selecting a portion of a speech segment lying or occurring in an intermediate frequency band near a low frequency portion of an aural bandwidth;

synthesizing harmonics of reconstructed speech using signals that lie or occur within the intermediate frequency band;

adjusting the gain of the synthesized harmonics by processing a correlation between the strength of the synthesized harmonics and the strength of the original speech signal;

filtering a portion of the adjusted synthesized harmonics based on a dynamic noise from changing noise conditions within a vehicle that is detected in the speech; and

weighting the filtered portion of the adjusted synthesized harmonics to reconstruct the speech segment lying in the intermediate frequency band.

19. The method that compensates for undesired changes in a speech segment of claim 18 where the act of weighting is based on multiple frequency responses that allow substantially all the frequencies bellows a plurality of specified frequencies to pass through.

20. The method that compensates for undesired changes in a speech segment of claim 18 where the act of weighting is based on a plurality of background noise estimates.

21. The method that compensates for undesired changes in a speech segment of claim 18 where the act of weighting is based on a plurality of linear modes.

22. The system of claim 1 , wherein the frequency response of the lowpass filter comprises a dynamic frequency response having a cutoff frequency that varies according to the dynamic noise.

Assignments (10)
NUNC PRO TUNC ASSIGNMENT Recorded Jun 19, 2023
From: BLACKBERRY LIMITED
To: MALIKIE INNOVATIONS LIMITED
Reel/Frame 064270/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 16, 2023
From: BLACKBERRY LIMITED
To: MALIKIE INNOVATIONS LIMITED
Reel/Frame 064104/0103 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 22, 2020
From: 2236008 ONTARIO INC.
To: BLACKBERRY LIMITED
Reel/Frame 053313/0315 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 4, 2014
From: QNX SOFTWARE SYSTEMS LIMITED
To: 8758271 CANADA INC.
Reel/Frame 032607/0943 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 4, 2014
From: 8758271 CANADA INC.
To: 2236008 ONTARIO INC.
Reel/Frame 032607/0674 →
CHANGE OF NAME Recorded Feb 27, 2012
From: QNX SOFTWARE SYSTEMS CO.
To: QNX SOFTWARE SYSTEMS LIMITED
Reel/Frame 027768/0863 →
CONFIRMATORY ASSIGNMENT Recorded Jul 9, 2010
From: QNX SOFTWARE SYSTEMS (WAVEMAKERS), INC.
To: QNX SOFTWARE SYSTEMS CO.
Reel/Frame 024659/0370 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Jun 3, 2010
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED; QNX SOFTWARE SYSTEMS (WAVEMAKERS), INC.; QNX SOFTWARE SYSTEMS GMBH & CO. KG
Reel/Frame 024483/0045 →
SECURITY AGREEMENT Recorded May 8, 2009
From: HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED; BECKER SERVICE-UND VERWALTUNG GMBH; CROWN AUDIO, INC.; HARMAN BECKER AUTOMOTIVE SYSTEMS (MICHIGAN), INC.; HARMAN BECKER AUTOMOTIVE SYSTEMS HOLDING GMBH; HARMAN BECKER AUTOMOTIVE SYSTEMS, INC.; HARMAN CONSUMER GROUP, INC.; HARMAN DEUTSCHLAND GMBH; HARMAN FINANCIAL GROUP LLC; HARMAN HOLDING GMBH & CO. KG; HARMAN MUSIC GROUP, INCORPORATED; HARMAN SOFTWARE TECHNOLOGY INTERNATIONAL BETEILIGUNGS GMBH; HARMAN SOFTWARE TECHNOLOGY MANAGEMENT GMBH; HBAS INTERNATIONAL GMBH; HBAS MANUFACTURING, INC.; INNOVATIVE SYSTEMS GMBH NAVIGATION-MULTIMEDIA; JBL INCORPORATED; LEXICON, INCORPORATED; MARGI SYSTEMS, INC.; QNX SOFTWARE SYSTEMS (WAVEMAKERS), INC.; QNX SOFTWARE SYSTEMS CANADA CORPORATION; QNX SOFTWARE SYSTEMS CO.; QNX SOFTWARE SYSTEMS GMBH; QNX SOFTWARE SYSTEMS GMBH & CO. KG; QNX SOFTWARE SYSTEMS INTERNATIONAL CORPORATION; QNX SOFTWARE SYSTEMS, INC.; XS EMBEDDED GMBH (F/K/A HARMAN BECKER MEDIA DRIVE TECHNOLOGY GMBH)
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 022659/0743 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 2, 2008
From: LI, XUEMAN; NONGPIUR, RAJEEV; LINSEISEN, FRANK; HETHERINGTON, PHILLIP A.
To: QNX SOFTWARE SYSTEMS (WAVEMAKERS), INC.
Reel/Frame 021030/0026 →