System and method for adapting natural language understanding (NLU) engines optimized on text to audio input

1 . A computer-implemented method, executed on a computing device, comprising:

generating a plurality of potential vocalizations of a plurality of text samples, wherein generating the plurality of potential vocalizations includes processing the plurality of text samples using a text-to-speech tokenizer that converts each text sample into a token representing the speech content of the respective text sample;

identifying a plurality of phonemes associated with the plurality of potential vocalizations, wherein identifying the plurality of phonemes associated with the plurality of potential vocalizations includes matching at least a portion of the plurality of potential vocalizations to the plurality of phonemes from a phonetic dictionary:

generating a plurality of phonetically-related text portions based upon, at least in part, the plurality of phonemes, wherein generating the plurality of phonetically-related text portions includes:

identifying a plurality of sequences of text portions with at least a threshold similarity to the plurality of phonemes; and

selecting a sequence of text portions from the plurality of sequences of text portions that covers each of the plurality of text samples and is in a same order as the plurality of text samples, thus defining the plurality of phonetically-related text portions; and

training a natural language understanding (NLU) engine using the plurality of phonetically-related text portions.

2 . The computer-implemented method of claim 1 , wherein the NLU engine is a machine learning model.

3 . The computer-implemented method of claim 1 , wherein training the NLU engine using the plurality of phonetically-related text portions includes generating a training batch including a predefined amount of the plurality of text samples and a predefined amount of the plurality of phonetically-related text portions.

4 . The computer-implemented method of claim 3 , wherein training the NLU engine using the plurality of phonetically-related text portions includes training the NLU engine with the training batch including the predefined amount of the plurality of text samples and the predefined amount of the plurality of phonetically-related text portions.

5 . A computing system comprising: a memory; and

a processor to generate a plurality of potential vocalizations of a plurality of text samples, wherein generating the plurality of potential vocalizations includes processing the plurality of text samples using a text-to-speech tokenizer that converts each text sample into a token representing the speech content of the respective text sample, to match at least a portion of the plurality of potential vocalizations to a plurality of phonemes from a phonetic dictionary, to generate a plurality of phonetically-related text portions based upon, at least in part, the plurality of phonemes, wherein generating the plurality of phonetically-related text portions includes: identifying a plurality of sequences of text portions with at least a threshold similarity to the plurality of phonemes, and selecting a sequence of text portions from the plurality of sequences of text portions that covers each of the plurality of text samples and is in a same order as the plurality of text samples, thus defining the plurality of phonetically-related text portions, and to train a natural language understanding (NLU) engine with a predefined amount of the plurality of text samples and a predefined amount of the plurality of phonetically-related text portions.

6 . The computing system of claim 5 , wherein the NLU engine is a machine learning model.

7 . The computing system of claim 6 , wherein generating the plurality of phonetically-related text portions includes generating the plurality of phonetically-related text portions at each epoch during fine-tuning of the NLU engine.

8 . The computing system of claim 5 , wherein training the NLU engine using the plurality of phonetically-related text portions includes generating a training batch including the predefined amount of the plurality of text samples and the predefined amount of the plurality of phonetically-related text portions.

9 . The computing system of claim 8 , wherein the predefined amount of the plurality of text samples is less than or equal to fifty percent of the training batch.

10 . A computer program product residing on a hardware computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:

generating a plurality of potential vocalizations of a plurality of text samples, wherein generating the plurality of potential vocalizations includes processing the plurality of text samples using a text-to-speech tokenizer that converts each text sample into a token representing the speech content of the respective text sample;

identifying a plurality of phonemes associated with the plurality of potential vocalizations, wherein identifying the plurality of phonemes associated with the plurality of potential vocalizations includes matching at least a portion of the plurality of potential vocalizations to the plurality of phonemes from a phonetic dictionary:

generating a plurality of phonetically-related text portions based upon, at least in part, the plurality of phonemes, wherein generating the plurality of phonetically-related text portions includes:

identifying a plurality of sequences of text portions with at least a threshold similarity to the plurality of phonemes; and

selecting a sequence of text portions from the plurality of sequences of text portions that covers each of the plurality of text samples and is in a same order as the plurality of text samples, thus defining the plurality of phonetically-related text portions; and

training a natural language understanding (NLU) engine using the plurality of phonetically-related text portions, wherein training the NLU engine using the plurality of phonetically-related text portions includes generating a training batch including a predefined amount of the plurality of text samples and a predefined amount of the plurality of phonetically-related text portions.

11 . The computer program product of claim 10 , wherein the NLU engine is a machine learning model.

12 . The computer program product of claim 11 , wherein generating the plurality of phonetically-related text portions includes generating the plurality of phonetically-related text portions at each epoch during fine-tuning of the NLU engine.