Granted Patent B2

US 12,367,867 · App. 17/973,266 · Granted Jul 22, 2025

Inventors: Sandeep Singh Spall (Moga, IN); Tarun Gupta (Noida, IN); Narang Lucky Manoharlal (Noida, IN)

Assignee: SAMSUNG ELECTRONICS CO., LTD.

G10L15/16G10L2015/025

Abstract

Claims (62)

1. A method of generating voice in a call session, the method comprising:

extracting a plurality of features from a voice input through an artificial neural network (ANN);

identifying one or more lost audio frames within the voice input, wherein the one or more lost audio frames are lost due to at least one of vocal issues or network issues;

predicting by the ANN, for each of the one or more lost audio frames, one or more features of the respective lost audio frame;

superposing the predicted features upon the voice input to generate an updated voice input; and

correcting the updated voice input by:

obtaining a confidence score of the updated voice input;

splitting the updated voice input into a plurality of phonemes based on the confidence score;

identifying one or more non-aligned phonemes out of the plurality of phonemes based on comparing the plurality of phonemes with language vocabulary knowledge;

generating a plurality of variant phonemes; and

updating the identified one or more non-aligned phonemes through one or more of:

replacing the identified one or more non-aligned phonemes with the plurality of variant phonemes;

adding additional phonemes to supplement the identified one or more non-aligned phonemes; or

deleting the identified one or more non-aligned phonemes.

2. The method of claim 1 , wherein the predicting by the ANN comprises:

operating a single-layer recurrent neural network for audio generation configured to predict raw audio samples.

3. The method of claim 2 , wherein the single-layer recurrent neural network comprises an architecture defined by WaveRNN and a dual softmax layer.

4. The method of claim 1 , further comprising:

updating the identified one or more non-aligned phonemes through regenerating the updated voice input defined by one or more of:

replacement of the identified one or more non-aligned phonemes with the variant phonemes;

removal of the identified one or more non-aligned phonemes; or

additional phonemes supplementing the identified one or more non-aligned phonemes.

5. The method of claim 1 , further comprising converting the updated voice input defined by a whisper into a converted updated voice input defined by a normal voice-input based on:

executing a time-alignment between whispered and a corresponding normal speech; and

learning by a generator adversarial network (GAN) model cross-domain relations between the whisper and the normal speech.

6. The method of claim 1 , further comprising:

improving voice quality of the updated voice input based on a nonlinear activation function.

7. A system for generating voice in a call session, the system comprising:

a microphone;

a memory storing one or more instructions; and

at least one processor configured to execute the one or more instructions to: extract a plurality of features from a voice input through an artificial neural network (ANN);

identify one or more lost audio frames within the voice input, wherein the one or more lost audio frames are lost due to at least one of vocal issues or network issues;

predict by the ANN the features of the one or more lost audio frames for each missing frame;

superpose the predicted features upon the voice input to generate an updated voice input; and

correct the updated voice input by:

obtaining a confidence score of the updated voice input;

splitting the updated voice input into a plurality of phonemes based on the confidence score;

identifying one or more non-aligned phonemes out of the plurality of phonemes based on comparing the plurality of phonemes with language vocabulary knowledge;

generating a plurality of variant phonemes; and

updating the identified one or more non-aligned phonemes through one or more of:

replacing the identified one or more non-aligned phonemes with the plurality of variant phonemes;

adding additional phonemes to supplement the identified one or more non-aligned phonemes; or

deleting the identified one or more non-aligned phonemes.

8. The system of claim 7 , wherein the processor is further configured to execute the one or more instructions to by the ANN by operating a single-layer recurrent neural network for audio generation configured to predict raw audio samples.

9. The system of claim 8 , wherein the single-layer recurrent neural network comprises an architecture defined by WaveRNN and a dual softmax layer.

10. The system of claim 7 , the processor is further configured to:

update the identified one or more non-aligned phonemes through regenerating the updated voice input defined by one or more of:

replacement of the identified one or more non-aligned phonemes with the variant phonemes;

removal of the identified one or more non-aligned phonemes; or

additional phonemes supplementing the identified one or more non-aligned phonemes.

11. The system of claim 7 , the processor is further configured to convert the updated voice input defined by a whisper into a converted updated voice input defined by a normal voice-input based on:

executing a time-alignment between whispered and a corresponding normal speech; and

learning by a generator adversarial network (GAN) model cross-domain relations between the whisper and the normal speech.

12. The system of claim 7 , the processor is further configured to improve voice quality of the updated voice input based on a nonlinear activation function.

13. The system of claim 7 , wherein the processor is further configured to:

extract one or more frames from the updated voice input;

encode a temporal information associated with the one or more frames through a convolution network;

execute a deconvolution over the encoded information to obtain a prediction score;

classify the extracted frames into key frames or non-key frames based on the prediction score; and

present a summary based on the classified key frames.

14. The system of claim 13 , the processor is configured to learn relation between raw audio frames (A) and a set of summary audios (S), wherein a distribution of resultant summary-audios F(A) is targeted to be similar to a distribution of the set of summary audios (S); and

train a summary discriminator to differentiate between the generated summary audio F(A) and a real summary audio.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (27)