Automatic learning of entities, words, pronunciations, and parts of speech
Systems for automatic speech recognition and/or natural language understanding automatically learn new words by finding subsequences of phonemes that, if they were a new word, would enable a successful tokenization of a phoneme sequence. Systems can learn alternate pronunciations of words by finding phoneme sequences with a small edit distance to existing pronunciations. Systems can learn the part of speech of words by finding part-of-speech variations that would enable parses by syntactic grammars. Systems can learn what types of entities a word describes by finding sentences that could be parsed by a semantic grammar but for the words not being on an entity list.
1. A computer-implemented method for automatically enhancing natural language recognition in an Automated Speech Recognition (ASR) system, the method comprising:
receiving digitized speech audio comprising one or more of a directly digitized audio waveform, a spectrogram and a spectrogram processed into mel filter bank bin values;
producing, via an acoustic model, a phoneme sequence based on the digitized speech audio;
generating a token sequence from the phoneme sequence via a pronunciation dictionary, wherein a token represents a word in the pronunciation dictionary;
identifying a phoneme subsequence from the phoneme sequence that does not match a token in the pronunciation dictionary;
identifying, via applying a semantic grammar to the token sequence, a slot for an entity where the phoneme subsequence fits in the semantic grammar, wherein the phoneme subsequence represents a new entity in the semantic grammar;
adding a new token to the pronunciation dictionary, the new token having the phoneme subsequence as its pronunciation; and
adding, to an entity list that is domain specific, the new entity with the phoneme subsequence as its pronunciation.
2. The computer-implemented method of claim 1 further comprising:
incrementing an occurrence count of the phoneme subsequence across a multiplicity of speech audio segments,
wherein the adding step is conditioned upon the occurrence count exceeding a threshold.
3. The computer-implemented method of claim 1 further comprising:
updating the token sequence probabilities of a statistical language model including the new entity.
4. A non-transitory computer readable medium comprising code that, if executed by a computer, would cause the computer to:
receiving digitized speech audio comprising one or more of a directly digitized audio waveform, a spectrogram and a spectrogram processed into mel filter bank bin values;
producing, via an acoustic model, a phoneme sequence based on the digitized speech audio;
generating a token sequence from the phoneme sequence via a pronunciation dictionary, wherein a token represents a word in the pronunciation dictionary;
identifying a phoneme subsequence from the phoneme sequence that does not match a token in the pronunciation dictionary;
identifying, via applying a semantic grammar to the token sequence, a slot for an entity where the phoneme subsequence fits in the semantic grammar, wherein the phoneme subsequence represents a new entity in the semantic grammar;
adding a new token to the pronunciation dictionary, the new token having the phoneme subsequence as its pronunciation; and
add, to an entity list that is domain specific, a new entity with the phoneme subsequence as its pronunciation.
5. The non-transitory computer readable medium of claim 4 further comprising causing the computer to:
increment an occurrence count of the phoneme subsequence across a multiplicity of speech audio segments,
wherein the adding step is conditioned upon the occurrence count exceeding a threshold.
6. The non-transitory computer readable medium of claim 4 further comprising causing the computer to:
update the token sequence probabilities of a statistical language model including the new entity.