Intonation transformation for speech therapy and the like
View Patent ↗The intonation of speech is modified by an appropriate combination of resampling and time-domain harmonic scaling. Resampling increases (upsampling) or decreases (downsampling) the number of data points in a signal. Harmonic scaling adds or removes pitch cycles to or from a signal. The pitch of a speech signal can be increased by combining downsampling with harmonic scaling that adds an appropriate number of pitch cycles. Alternatively, pitch can be decreased by combining upsampling with harmonic scaling that removes an appropriate number of pitch cycles. The present invention can be implemented in an automated speech-therapy tool that is able to modify the intonation of prerecorded reference speech signals for playback to a user to emphasize the correct pronunciation by increasing the pitch of selected portions of words or phrases that the user had previously mispronounced.
1. A method for generating an output audio signal from an input audio signal having a number of pitch cycles, each input pitch cycle represented by a plurality of data points, the method comprising a combination of resampling and harmonic scaling, wherein:
the resampling comprises changing the number of data points in an audio signal, wherein the resampling comprises an upsampling phase followed by a downsampling phase to achieve a desired resampling ratio, wherein:
the upsampling phase comprises upsampling the audio signal based on an upsampling rate value to generate an upsampled signal; and
the downsampling phase comprises downsampling the upsampled signal based on a downsampling rate value selected to achieve, in combination with the upsampling phase, the desired resampling ratio; and
the harmonic scaling comprises changing the number of pitch cycles in an audio signal, wherein the output audio signal has a pitch that is different from the pitch of the input audio signal.
2. The invention of claim 1 , wherein the harmonic scaling is implemented before the resampling.
3. The invention of claim 1 , wherein the number of data points in the output audio signal is the same as the number of data points in the input audio signal.
4. The invention of claim 1 , further comprising changing the timing of the input audio signal, wherein the number of data points in the output audio signal is different from the number of data points in the input audio signal.
5. The invention of claim 1 , further comprising changing the volume of the input audio signal.
6. The invention of claim 1 , wherein the method is implemented to modify the intonation of speech corresponding to the input audio signal.
7. The invention of claim 6 , wherein the method is implemented as part of a computer-implemented tool that modifies the intonation of one or more reference words or phrases played to a user of the tool.
8. The invention of claim 7 , wherein the computer-implemented tool is a speech therapy tool.
9. The invention of claim 1 , further comprising:
comparing a user speech signal to a reference speech signal to select one or more parts of the reference speech signal to emphasize;
applying the combination of resampling and harmonic scaling to change the pitch of the one or more selected parts of the reference speech signal to generate an intonation-transformed speech signal; and
playing the intonation-transformed speech signal to the user.
10. The invention of claim 1 , wherein the desired resampling ratio has a value other than one.
11. A machine-readable medium, having encoded thereon program code, wherein, when the program code is executed by a machine, the machine implements a method for generating an output audio signal from an input audio signal having a number of pitch cycles, each input pitch cycle represented by a plurality of data points, the method comprising a combination of resampling and harmonic scaling, wherein:
the resampling comprises changing the number of data points in an audio signal, wherein the resampling comprises an upsampling phase followed by a downsampling phase to achieve a desired resampling ratio, wherein:
the upsampling phase comprises upsampling the audio signal based on an upsampling rate value to generate an upsampled signal; and
the downsampling phase comprises downsampling the upsampled signal based on a downsampling rate value selected to achieve, in combination with the upsampling phase, the desired resampling ratio; and
the harmonic scaling comprises changing the number of pitch cycles in an audio signal, wherein the output audio signal has a pitch that is different from the pitch of the input audio signal.
12. A computer-implemented method comprising:
comparing a user speech signal to a reference speech signal to select one or more parts of the reference speech signal to emphasize;
processing the one or more selected parts of the reference speech signal to generate an intonation-transformed speech signal, wherein generating the intonation-transformed speech signal comprises applying a combination of resampling and harmonic scaling to change the pitch of the one or more selected parts of the reference speech signal, wherein:
the resampling comprises changing the number of data points in an audio signal; and
the harmonic scaling comprises changing the number of pitch cycles in an audio signal; and
playing the intonation-transformed speech signal to the user.
13. The invention of claim 12 , wherein the harmonic scaling is implemented before the resampling.
14. The invention of claim 12 , wherein the number of data points in the output audio signal is the same as the number of data points in the input audio signal.
15. The invention of claim 12 , further comprising changing the timing of the input audio signal, wherein the number of data points in the output audio signal is different from the number of data points in the input audio signal.
16. The invention of claim 12 , further comprising changing the volume of the input audio signal.
17. The invention of claim 12 , wherein the resampling comprises an upsampling phase followed by a downsampling phase to achieve a desired resampling ratio, wherein:
the upsampling phase comprises upsampling the audio signal based on an upsampling rate value to generate an upsampled signal; and
the downsampling phase comprises downsampling the upsampled signal based on a downsampling rate value selected to achieve, in combination with the upsampling phase, the desired resampling ratio.
18. The invention of claim 17 , wherein the desired resampling ratio has a value other than one.
19. The invention of claim 12 , wherein the method is implemented as part of a computer-implemented tool that modifies the intonation of one or more reference words or phrases played to a user of the tool.
20. A machine-readable medium, having encoded thereon program code, wherein, when the program code is executed by a machine, the machine implements a method comprising:
comparing a user speech signal to a reference speech signal to select one or more parts of the reference speech signal to emphasize;
processing the one or more selected parts of the reference speech signal to generate an intonation-transformed speech signal, wherein generating the intonation-transformed speech signal comprises applying a combination of resampling and harmonic scaling to change the pitch of the one or more selected parts of the reference speech signal, wherein:
the resampling comprises changing the number of data points in an audio signal; and
the harmonic scaling comprises changing the number of pitch cycles in an audio signal; and
playing the intonation-transformed speech signal to the user.