Methods and apparatus to convert image to audio

1 . An apparatus comprising:

at least one memory;

instructions; and

processor circuitry to execute the instructions to:

identify a word in an image, the word to be converted to an audio waveform;

encode the word identified in the image into an ordered list of phonemes;

pad the ordered list of phonemes with one or more placeholders to generate a fixed-length representation of the word identified in the image;

determine, at a neural network, a duration that a phoneme of the ordered list of phonemes is to be expressed in the audio waveform, wherein the duration is a duration value;

generate an expanded representation of the fixed-length representation based on the duration value and a dimensionality; and

synthesize the audio waveform of the word based on the expanded representation.

2 . The apparatus of claim 1 , wherein the duration value is a first duration value, and the one or more placeholders includes a symbol that is associated with a second duration value of zero.

3 . The apparatus of claim 1 , wherein the processor circuitry is to execute the instructions to generate a mel-spectrogram from the expanded representation.

4 . The apparatus of claim 1 , the processor circuitry is a neural network accelerator that is independent of an operating system with which the neural network accelerator is associated.

5 . The apparatus of claim 1 , wherein the processor circuitry is to train an encoding portion of the neural network based on cross-entropy loss.

6 . The apparatus of claim 1 , wherein the processor circuitry is to execute the instructions to train a decoding portion of the neural network based on a structural similarity loss.

7 . The apparatus of claim 1 , wherein the processor circuitry is to execute the instructions to train a duration prediction portion of the neural network based on mean square error.

8 . A non-transitory computer readable storage medium comprising instructions which, when executed by processor circuitry, cause the processor circuitry to:

identify a word in an image, the word to be converted to an audio waveform;

encode the word identified in the image into an ordered list of phonemes;

pad the ordered list of phonemes with one or more placeholders to generate a fixed-length representation of the word identified in the image;

determine, at a neural network, a duration that a phoneme of the ordered list of phonemes is to be expressed in the audio waveform, wherein the duration is a duration value;

generate an expanded representation of the fixed-length representation based on the duration value and a dimensionality; and

synthesize the audio waveform of the word based on the expanded representation.

9 . The non-transitory computer readable medium of claim 8 , wherein the duration value is a first duration value, and wherein the one or more placeholders includes a symbol that is associated with a second duration value of zero.

10 . The non-transitory computer readable medium of claim 8 , wherein the instructions, when executed, cause the processor circuitry to generate a mel-spectrogram from the expanded representation.

11 . The non-transitory computer readable medium of claim 8 , wherein the processor circuitry is a neural network accelerator that is independent of an operating system with which the neural network accelerator is associated.

12 . The non-transitory computer readable medium of claim 8 , wherein the instructions, when executed, cause the processor circuitry to train an encoding portion of the neural network based on cross-entropy loss.

13 . The non-transitory computer readable medium of claim 8 , wherein the instructions, when executed, cause the processor circuitry to train a decoding portion of the neural network based on a structural similarity loss.

14 . A method comprising:

identifying, by executing an instruction with processor circuitry, a word in an image, the word to be converted to an audio waveform;

encoding, by executing an instruction with the processor circuitry, the word identified in the image into an ordered list of phonemes;

pad the ordered list of phonemes with one or more placeholders to generate a fixed-length representation of the word identified in the image;

determine, at a neural network, a duration that a phoneme of the ordered list of phonemes is to be expressed in the audio waveform, wherein the duration is a duration value;

generate an expanded representation of the fixed-length representation based on the duration and a dimensionality; and

synthesizing, by executing an instruction with the processor circuitry, the audio waveform of the word based on the expanded representation.

15 . The method of claim 14 , wherein the duration value is a first duration value, and the one or more placeholders includes a symbol that is associated with a second duration value of zero.

16 . The method of claim 14 , wherein the synthesizing of the audio waveform of the word based on the expanded representation includes generating a mel-spectrogram.

17 . The method of claim 14 , wherein the processor circuitry is a neural network accelerator that is independent of an operating system with which the neural network accelerator is associated.

18 . The method of claim 14 , further including training an encoding portion of the neural network based on cross-entropy loss.

19 . The method of claim 14 , further including training a decoding portion of the neural network based on a structural similarity loss.

20 . The method of claim 14 , further including training a duration prediction portion of the neural network based on mean square error.