Granted Patent B2

US 12,347,452 · App. 17/542,823 · Granted Jul 1, 2025

Inventors: Marc Adrian Chua Lihan (Tokyo, JP); Mao-Yuan Kao (Tokyo, JP); Jing-Ya Huang (Tokyo, JP); Cheng-Ho Chen (Tokyo, JP); Kai-Ju Chang (Tokyo, JP)

Assignee: LY Corporation

G10L25/63G06F3/16G10L15/00H04L51/04H04L51/10

Abstract

Claims (48)

1. A method for generating multimedia content, the method comprising:

receiving, by a first user terminal, voice data that comprises a recording of a voice of a user, wherein the voice data comprises first data corresponding to a first time section in which a signal strength is less than a predetermined threshold and second data corresponding to a second time section in which the signal strength is equal to or greater than the predetermined threshold;

identifying an emotional state of the user based on an audio frequency characteristic of the voice data;

receiving, by the first user terminal, a selection of one character from among a plurality of characters from the user;

generating a multimedia content comprising the voice data and an animated graphic object of the selected one character expressing the emotional state of the user, such that the animated graphic object is maintained in a stationary state during the first time section, and is replayed during the second time section; and

transmitting the multimedia content to a second user terminal of another user.

2. The method according to claim 1 , wherein the transmitting comprises transmitting the multimedia content to the second user terminal through a chat room on an instant messaging application.

3. The method according to claim 1 , wherein the selected one character is associated with a plurality of animated graphic objects, each of which expresses a different emotional state, and

the multimedia content comprises the voice data and the plurality of animated graphic objects.

4. The method according to claim 1 , further comprising identifying an action of the selected one character based on the emotional state of the user.

5. A method for generating multimedia content, the method comprising:

receiving, by a first user terminal, voice data that comprises a recording of a voice of a user, wherein the voice data comprises first data corresponding to a first time section associated with a first emotional state of the user and second data corresponding to a second time section associated with a second emotional state of the user;

identifying the first emotional state and the second emotional state based on an audio frequency characteristic of the voice data;

receiving, by the first user terminal, a selection of one character from among a plurality of characters from the user, wherein the selected one character is associated with a first animated graphic object expressing the first emotional state and a second animated graphic object expressing the second emotional state;

generating multimedia content comprising the voice data, the first animated graphic object, and the second animated graphic object, such that the multimedia content replays the first animated graphic object and the first data together during the first time section, and replays the second animated graphic object and the second data together during the second time section; and

transmitting the multimedia content to a second user terminal of another user.

6. The method according to claim 1 , further comprising:

performing a voice recognition process based on the voice data to obtain a character string; and

identifying the emotional state of the user based on the character string.

7. The method according to claim 1 , further comprising displaying the plurality of characters on a display, wherein the plurality of characters comprise the animated graphic object expressing the emotional state of the user.

8. The method according to claim 7 , wherein the plurality of characters are arranged and displayed based on a usage history.

9. The method according to claim 7 , wherein the plurality of characters are recommendations for characters frequently used by other users in relation to the emotional state of the user.

10. An information processing system comprising:

a communication interface;

at least one memory; and

at least one processor connected to the at least one memory and configured to execute at least one program stored in the at least one memory, wherein the at least one program comprises instructions for controlling the information processing system to:

receive voice data that comprises a recording of a voice of a first user from a first user terminal, wherein the voice data comprises first data corresponding to a first time section in which a signal strength is less than a predetermined threshold and second data corresponding to a second time section in which the signal strength is equal to or greater than the predetermined threshold;

identify an emotional state of the first user based on an audio frequency characteristic of the voice data;

receive, from the first user terminal, a selection of one character from among a plurality of characters;

generate a multimedia content comprising the voice data and an animated graphic object of the selected one character expressing the emotional state of the first user, such that the animated graphic object is maintained in a stationary state during the first time section, and is replayed during the second time section; and

transmit the multimedia content to a second user terminal associated with a second user.

11. The information processing system according to claim 10 , wherein the second user is included in a chat room on a same instant messaging application as the first user.

12. The information processing system according to claim 10 , wherein the selected one character is associated with a plurality of animated graphic objects, each of which expresses a different emotional state, and

the animated graphic object of the multimedia content is associated with the voice data and the emotional state of the first user.

13. The information processing system according to claim 10 , wherein the at least one program further includes instructions for controlling the information processing system to identify an action of the selected one character included in the multimedia content based on the emotional state of the first user.

14. An information processing system comprising:

a communication interface;

at least one memory; and

at least one processor connected to the at least one memory and configured to execute at least one program stored in the at least one memory, wherein the at least one program comprises instructions for controlling the information processing system to:

receive voice data that comprises a recording of a voice of a first user from a first user terminal, wherein the voice data comprises first data corresponding to a first time section associated with a first emotional state of the first user and second data corresponding to a second time section associated with a second emotional state of the first user;

identify an emotional state of the first user based on an audio frequency characteristic of the voice data;

receive, from the first user terminal, a selection of one character from among a plurality of characters, wherein the selected one character is associated with a first animated graphic object expressing the first emotional state and a second animated graphic object expressing the second emotional state;

generate a multimedia content comprising the voice data, the first animated graphic object, and the second animated graphic object, such that the multimedia content replays the first animated graphic object and the first data together during the first time section, and replays the second animated graphic object and the second data together during the second time section; and

transmit the multimedia content to a second user terminal associated with a second user.

15. The information processing system according to claim 10 , wherein the at least one program further includes instructions for controlling the information processing system to analyze the audio frequency characteristic of the voice data and identify the emotional state of the first user based on the audio frequency characteristic, without considering language and content.

16. The information processing system according to claim 10 , wherein the at least one program further includes instructions for controlling the information processing system to:

perform a voice recognition process based on the voice data to obtain a character string; and

identify the emotional state of the first user based on the character string.

Assignments (3)

Priority Claims (1)

Continuity (1)

References Cited (14)