Delivering fundamental frequency and amplitude envelope cues to enhance speech understanding
Systems, apparatus, and techniques are described to perform operations including receiving speech that includes audio signals from a talker. A fundamental frequency, unique to the talker, is extracted from the audio signals. A tone in frequency with the extracted fundamental frequency of the received speech is modulated and the modulated tone is delivered to an audio transducing device.
1. A computer-implemented method to enhance speech understanding, the method comprising:
receiving speech from a talker at a system including one or more computers, the speech including audio signals;
extracting, at the system, a fundamental frequency from the audio signals, wherein the extracted fundamental frequency is unique to the talker;
modulating, at the system, a tone in frequency with the extracted fundamental frequency of the received speech; and
delivering the modulated tone as an acoustic signal.
2. The method of claim 1 , further comprising:
modulating, at the system, the tone in amplitude with an amplitude envelope of the received speech;
providing the tone of the speech, modulated in frequency with the extracted fundamental frequency of the received speech, and in amplitude with the amplitude envelope of the received speech,
wherein the modulated tone is amplified prior to the delivering.
3. The method of claim 2 , wherein the amplitude envelope is obtained by half-wave rectification and low-pass filtering.
4. The method of claim 1 , further comprising modulating the tone of the speech in frequency with dynamic changes of the extracted fundamental frequency.
5. The method of claim 4 , wherein the dynamic changes of the extracted fundamental frequency are obtained using YIN algorithm.
6. A computer-readable medium tangibly encoding software instructions executable by one or more data processing apparatus to perform operations comprising:
receiving speech from a talker, the speech including audio signals;
extracting a fundamental frequency from the audio signals, wherein the extracted fundamental frequency is unique to the talker;
modulating a tone in frequency with the extracted fundamental frequency of the received speech;
amplifying the modulated tone; and
delivering the modulated tone as an acoustic signal.
7. The computer-readable medium of claim 6 , further comprising:
modulating the tone of the speech in amplitude with an amplitude envelope of the received speech; and
providing the tone, modulated in amplitude with the amplitude envelope of the received speech.
8. The computer-readable medium of claim 7 , wherein the amplitude envelope is obtained by half-wave rectification and low-pass filtering.
9. The computer-readable medium of claim 6 , further comprising modulating the tone of the speech in frequency with dynamic changes of the extracted fundamental frequency.
10. The computer readable medium of claim 9 , wherein the dynamic changes of the extracted fundamental frequency are obtained using YIN algorithm.
11. A cochlear implant system comprising:
a cochlear implant;
means for receiving speech from a talker, the speech including audio signals;
means for extracting a fundamental frequency from the audio signals, wherein the extracted fundamental frequency is unique to the talker;
means for modulating a tone in frequency with the extracted fundamental frequency of the received speech; and
means for providing the modulated tone to the cochlear implant as an acoustic signal.
12. The system of claim 11 , further comprising means for:
modulating the tone in amplitude with an amplitude envelope of the received speech; and
providing the tone of the speech, modulated in amplitude with the amplitude envelope.
13. The system of claim 12 , wherein the amplitude envelope is obtained by half-wave rectification and low-pass filtering of the received speech.
14. The system of claim 11 , further comprising means for modulating the tone in frequency with dynamic changes to the extracted fundamental frequency.
15. The system of claim 14 , wherein the dynamic changes of the extracted fundamental frequency are obtained using YIN algorithm.
16. A system comprising:
a first transformer to receive audio signals and transform the received audio signals into a frequency-domain representation;
a processor operatively coupled to the transformer, the processor including:
a frequency compression unit to perform frequency compression of the frequency-domain representation of the audio signal,
a frequency transposition unit operatively coupled to the frequency compression unit to perform frequency transposition of the frequency-domain representation of the audio signal on which the frequency compression has been performed;
a second transformer operatively coupled to the processor to transform the compressed and transposed audio signals into a time-domain representation; and
a transmitter operatively coupled to the second transformer to deliver the signals in the time-domain representation as acoustic signals.
17. The system of claim 16 , further comprising an amplifier operatively coupled to the second transformer to amplify the signals in the time-domain representation.
18. The system of claim 17 , further comprising a level control unit operatively coupled to the amplifier to control a level by which the amplifier amplifies the signals in the time-domain representation.
19. The system of claim 16 , wherein the frequency compression is applied to the frequency region in which the fundamental frequency of most talkers occurs.
20. The system of claim 16 , wherein the frequency transposition is applied such that the frequency region in which the fundamental frequency of most talkers occurs is audible to the user.
21. The system of claim 16 , further comprising an audio transducing device to receive the signals in the time-domain representation delivered by the second transformer.