Speech recognition apparatus and method
According to one embodiment, a speech recognition apparatus includes processing circuitry. The processing circuitry generates a plurality of augmented speech data, based on input speech data, generates a plurality of acoustic scores, based on the plurality of augmented speech data and an acoustic model, generates a plurality of adjusted acoustic scores by resampling the acoustic scores, generates an integrated acoustic score by integrating the adjusted acoustic scores, generates an integrated lattice, based on the integrated acoustic score, a pronunciation dictionary, and a language model, and searches a speech recognition result with a highest likelihood from the integrated lattice.
1. A speech recognition apparatus comprising processing circuitry configured to:
generate a plurality of augmented speech data, based on input speech data, the plurality of augmented speech data each corresponding to an entirety of the input speech data,
wherein at least one of the plurality of augmented speech data is generated by applying a signal processing operation selected from:
(i) a speech speed conversion in which the input speech data is resampled at a sampling rate different from an original sampling rate and then restored to the original sampling rate using interpolation,
(ii) a sound volume conversion in which an amplitude of a waveform of the input speech data is multiplied by a gain coefficient, and
(iii) a voice quality conversion in which a pitch of the input speech data is altered using a Pitch Synchronous Overlap and Add (PSOLA) method;
generate a plurality of acoustic scores, based on the plurality of augmented speech data and an acoustic model;
generate a plurality of adjusted acoustic scores by resampling the acoustic scores, wherein the resampling comprises, for each acoustic score sequence of K-dimensional vectors over T n time frames, resampling each of the K dimensions independently using interpolation at a sampling rate of T/T n , where T n is a number of time frames of the plurality of augmented speech data, and where T is a number of time frames of the input speech data, thereby converting each acoustic score sequence into a corresponding sequence over T time frames;
generate an integrated acoustic score by calculating, for each time frame, an average value, median, or maximum value across the adjusted acoustic scores;
generate an integrated lattice, based on the integrated acoustic score, a pronunciation dictionary, and a language model;
search a speech recognition result with a highest likelihood from the integrated lattice;
estimate a conversion parameter based on the input speech data, wherein the conversion parameter is at least one of (i) a speech speed conversion parameter based on a comparison between a number of morae per unit time and a reference speech speed, (ii) a sound volume conversion parameter based on a power spectrum comparison, and (iii) a voice quality conversion parameter based on a pitch mean comparison; and
generate at least one of the plurality of augmented speech data by executing a conversion process in which the estimated conversion parameter is uniformly applied to the input speech data using the corresponding signal processing operation.
2. The speech recognition apparatus according to claim 1 , wherein the integrated lattice is a word lattice in which candidate words by speech recognition are nodes, and likelihoods of the candidate words are edges.
3. The speech recognition apparatus according to claim 1 , wherein the processing circuitry is further configured to auto-determine a conversion parameter relating to the conversion process, based on the input speech data.
4. The speech recognition apparatus according to claim 1 , wherein the plurality of augmented speech data include the input speech data.
5. The speech recognition apparatus according to claim 1 , wherein the acoustic model is a single model that is trained to output a posterior probability corresponding to an acoustic score by inputting speech data in units of at least one of a phoneme, a syllable, a character, a word-piece, and a word.
6. The speech recognition apparatus according to claim 1 , wherein the processing circuitry is further configured to generate an adapted acoustic model in which the acoustic model is adapted to a speaker of the input speech data, based on the input speech data and the speech recognition result corresponding to the input speech data.
7. A speech recognition apparatus comprising processing circuitry configured to:
generate a plurality of augmented speech data, based on input speech data, the plurality of augmented speech data each corresponding to an entirety of the input speech data,
wherein at least one of the plurality of augmented speech data is generated by applying a signal processing operation selected from:
(i) a speech speed conversion in which the input speech data is resampled at a sampling rate different from an original sampling rate and then restored to the original sampling rate using interpolation,
(ii) a sound volume conversion in which an amplitude of a waveform of the input speech data is multiplied by a gain coefficient, and
(iii) a voice quality conversion in which a pitch of the input speech data is altered using a Pitch Synchronous Overlap and Add (PSOLA) method;
generate a plurality of acoustic scores, based on the plurality of augmented speech data and an acoustic model;
generate a plurality of adjusted acoustic scores by resampling the acoustic scores, wherein the resampling comprises, for each acoustic score sequence of K-dimensional vectors over T n time frames, resampling each of the K dimensions independently using interpolation at a sampling rate of T/T n , where T n is a number of time frames of the plurality of augmented speech data, and where T is a number of time frames of the input speech data, thereby converting each acoustic score sequence into a corresponding sequence over T time frames;
generate a plurality of lattices, based on the acoustic scores, a pronunciation dictionary, and a language model;
generate an integrated lattice by integrating the lattices;
search a speech recognition result with a highest likelihood from the integrated lattice;
estimate a conversion parameter based on the input speech data, wherein the conversion parameter is at least one of (i) a speech speed conversion parameter based on a comparison between a number of morae per unit time and a reference speech speed, (ii) a sound volume conversion parameter based on a power spectrum comparison, and (iii) a voice quality conversion parameter based on a pitch mean comparison; and
generate at least one of the plurality of augmented speech data by executing a conversion process in which the estimated conversion parameter is uniformly applied to the input speech data using the corresponding signal processing operation.
8. The speech recognition apparatus according to claim 7 , wherein each of the lattices is a word lattice in which candidate words by speech recognition are nodes, and likelihoods of the candidate words are edges.
9. The speech recognition apparatus according to claim 8 , wherein the processing circuitry is further configured to generate the integrated lattice by connecting start points of the lattices and connecting end points of the lattices, and integrating common parts of the candidate words.
10. The speech recognition apparatus according to claim 7 , wherein the processing circuitry is further configured to auto-determine a conversion parameter relating to the conversion process, based on the input speech data.
11. The speech recognition apparatus according to claim 7 , wherein the processing circuitry is further configured to generate an adapted acoustic model in which the acoustic model is adapted to a speaker of the input speech data, based on the input speech data and the speech recognition result corresponding to the input speech data.
12. A speech recognition method comprising:
generating a plurality of augmented speech data, based on input speech data, the plurality of augmented speech data each corresponding to an entirety of the input speech data,
wherein at least one of the plurality of augmented speech data is generated by applying a signal processing operation selected from:
(i) a speech speed conversion in which the input speech data is resampled at a sampling rate different from an original sampling rate and then restored to the original sampling rate using interpolation,
(ii) a sound volume conversion in which an amplitude of a waveform of the input speech data is multiplied by a gain coefficient, and
(iii) a voice quality conversion in which a pitch of the input speech data is altered using a Pitch Synchronous Overlap and Add (PSOLA) method;
generating a plurality of acoustic scores, based on the plurality of augmented speech data and an acoustic model;
generating a plurality of adjusted acoustic scores by resampling the acoustic scores, wherein the resampling comprises, for each acoustic score sequence of K-dimensional vectors over T n time frames, resampling each of the K dimensions independently using interpolation at a sampling rate of T/T n , where T n is a number of time frames of the plurality of augmented speech data, and where T is a number of time frames of the input speech data, thereby converting each acoustic score sequence into a corresponding sequence over T time frames;
generating an integrated acoustic score by calculating, for each time frame, an average value, median, or maximum value across the adjusted acoustic scores;
generating an integrated lattice, based on the integrated acoustic score, a pronunciation dictionary, and a language model; and
searching a speech recognition result with a highest likelihood from the integrated lattice;
estimating a conversion parameter based on the input speech data, wherein the conversion parameter is at least one of (i) a speech speed conversion parameter based on a comparison between a number of morae per unit time and a reference speech speed, (ii) a sound volume conversion parameter based on a power spectrum comparison, and (iii) a voice quality conversion parameter based on a pitch mean comparison; and
generating at least one of the plurality of augmented speech data by executing a conversion process in which the estimated conversion parameter is uniformly applied to the input speech data using the corresponding signal processing operation.