Apparatus and method for generating avatar based video message

1. An apparatus for generating an avatar based video message, the apparatus comprising:

an audio input unit configured to receive speech of a user;

a user input unit configured to receive input from the user;

a display unit configured to output display information; and

a control unit configured to perform speech recognition based on the speech of the user to generate a word sequence of the speech of the user, to generate editing information comprising the word sequence divided into a plurality of editable units based on a measured energy of the speech of the user, to generate avatar animation that moves based on the word sequence, and to generate an avatar based video message that vocalizes the word sequence of the speech of the user and that displays the avatar animation such that the avatar animation moves in synchronization with the vocalized word sequence.

2. The apparatus of claim 1 , wherein the display unit is configured to display the editing information to the user, the editing information comprising the word sequence converted from the recognized speech and synchronization information for speech sections corresponding to respective words included in the word sequence.

3. The apparatus of claim 2 , wherein the control unit is further configured to output information indicating the plurality of editable units through the display unit.

4. The apparatus of claim 3 , wherein the information indicating the plurality of editable units comprises visual indication information that is used to display the word sequence such that the word sequence is differentiated into units of editable words.

5. The apparatus of claim 4 , wherein the control unit is further configured to control the display unit such that a cursor serving as the visual indication information moves in steps of editable units in the word sequence.

6. The apparatus of claim 3 , wherein the control unit is further configured to edit the word sequence at an editable unit according to a user input signal.

7. The apparatus of claim 3 , wherein the control unit is further configured to determine, as an editable unit, a location of a boundary that is positioned among speech sections corresponding to the respective words of the word sequence and which comprises an energy below a predetermined threshold value.

8. The apparatus of claim 2 , wherein

the control unit is further configured to calculate a linked sound score that refers to an extent to which at least two words included in the word sequence are recognized as linked sounds;

the control unit is further configured to calculate a clear sound score that refers to an extent to which the at least two words are recognized as a clear sound; and

if a value obtained by subtracting the clear score from the linked score is below a predetermined threshold value, the control unit is further configured to:

determine that the at least two words are vocalized as a clear sound; and

determine, as the editable location, a location corresponding to a boundary between the at least two words determined as the clear sound.

9. The apparatus of claim 1 , wherein the control unit is further configured to edit the speech based on an editing action that comprises at least one of a deletion, a replacement, and an insertion, the deletion action deleting at least one word included in the word sequence, the replacement action replacing at least one word included in the word sequence with one or more other words, and the insertion action inserting one or more new words into the word sequence.

10. The apparatus of claim 1 , wherein the control unit comprises a silence duration corrector configured to shorten a section of silence included in new speech that is input to modify at least one word included in the word sequence or to insert a new word into the word sequence.

11. The apparatus of claim 1 , wherein the control unit is further configured to generate the editing information comprising the word sequence divided into the plurality of editable units based on clear sounds and based on linked sounds.

12. A method of generating an avatar based video message, the method comprising:

receiving speech input by a user;

performing speech recognition on the input speech to generate a word sequence of the speech input by the user;

generating editing information comprising the word sequence divided into a plurality of editable units based on a measured energy of the speech of the user

generating avatar animation that moves based on the word sequence; and

generating an avatar based video message that vocalizes the word sequence of the speech of the user and that displays the avatar animation such that the avatar animation moves in synchronization with the vocalized word sequence.

13. The method of claim 12 , further comprising:

displaying the editing information,

wherein the editing information comprises the word sequence converted from the speech and synchronization information for speech sections corresponding to respective words included in the word sequence.

14. The method of claim 13 , further comprising editing the word sequence, wherein the editing of the word sequence comprises:

displaying information indicating the plurality of editable units; and

editing the word sequence at an editable unit that is selected according to a user input signal.

15. The method of claim 14 , wherein the information indicating the plurality of editable units comprises visual indication information that is used to display the word sequence such that the word sequence is differentiated into units of editable words.

16. The method of claim 14 , wherein the editable unit represents a location of a boundary that is positioned among speech sections corresponding to the respective words of the word sequence and which comprises an energy below a predetermined threshold value.

17. The method of claim 14 , further comprising:

subtracting a clear sound score that refers to an extent to which at least two words included in the word sequence are recognized as a clear sound, from a linked sound score that refers to an extent to which the at least two words are recognized as a linked sound; and

if the subtraction value is below a predetermined threshold value, determining that the at least two words are vocalized as a clear sound and determining, as the editable location, a location corresponding to a boundary between the at least two words determined as the clear sound.

18. The method of claim 12 , wherein the speech is edited based on at least one editing action that comprises at least one of a deletion, a replacement, and an insertion, the deletion action deleting at least one word included in the word sequence, the replacement action replacing at least one word included in the word sequence with one or more other words, and the insertion action inserting one or more new words into the word sequence.

19. The method of claim 12 , further comprising editing the word sequence,

wherein the editing of the word sequence comprises shortening a section of silence included in new speech that is input to modify at least one word included in the word sequence or to insert a new word into the word sequence.