Electronic device, avatar facial expression system and controlling method thereof

1. An electronic device comprising:

a sensor part that is attached in the facial part of a user and detects a facial electromyography signal according to a facial expression;

a processor that extracts a feature based on the facial electromyography signal and determines a data value of an avatar's facial movement corresponding to the extracted feature; and

a display that controls a facial movement of the avatar based on the data value of the avatar's facial movement and displays a facial expression of the avatar corresponding to the facial expression of the user,

wherein the processor is configured to:

partition features corresponding to predetermined facial expression data into a plurality of groups based on similarity, and determine a data value of the avatar's facial movement based on a learning model having the highest similarity to a feature corresponding to the user's predetermined facial expression among a plurality of learning models trained with respect to each of the plurality of partitioned groups,

wherein the processor is configured to:

calculate a distance between a center value of a group including a feature corresponding to predetermined facial expression data included in each of the plurality of learning models and a feature corresponding to the user's predetermined facial expression, and determine a learning model including a group for which the distance is the closest as the learning model having the highest similarity.

2. The electronic device of claim 1 ,

wherein, in the sensor part,

a plurality of sensors are arranged as a sensor set in each of the upper and lower portions of the left eye of the user, and a plurality of sensors are arranged as a sensor set in each of the upper and lower portions of the right eye of the user.

3. The electronic device of claim 2 ,

wherein the processor is configured to:

acquire a difference among a plurality of signals detected at the plurality of sensors included in each of the sensor sets as a facial electromyography signal of each of the sensor sets.

4. The electronic device of claim 1 ,

wherein the processor is configured to:

filter the facial electromyography signal based on a band pass filter in a band of 20 Hz to 450 Hz.

5. The electronic device of claim 1 ,

wherein the processor is configured to:

extract the feature while moving a window in a predetermined size at a predetermined time interval in the facial electromyography signal.

6. The electronic device of claim 1 ,

wherein the feature includes at least one of a root mean square (RMS), a cepstral coefficient (CC), sample entropy (SE), or a length of a signal (WL).

7. A controlling method of an electronic device comprising:

detecting by a sensor part that is attached in the facial part of a user a facial electromyography signal according to a facial expression;

extracting a feature based on the facial electromyography signal;

determining a data value of an avatar's facial movement corresponding to the extracted feature; and

controlling a facial movement of the avatar based on the data value of the avatar's facial movement and displaying a facial expression of the avatar corresponding to the facial expression of the user,

wherein the determining a data value of a facial movement comprises:

partitioning features corresponding to predetermined facial expression data into a plurality of groups based on similarity, and determining a data value of the avatar's facial movement based on a learning model having the highest similarity to a feature corresponding to the user's predetermined facial expression among a plurality of learning models trained with respect to each of the plurality of partitioned groups, and

wherein the determining a data value of a facial movement comprises:

calculating a distance between a center value of a group including a feature corresponding to predetermined facial expression data included in each of the plurality of learning models and a feature corresponding to the user's predetermined facial expression, and determining a learning model including a group for which the distance is the closest as the learning model having the highest similarity.

8. The controlling method of an electronic device of claim 7 ,

wherein the sensor part includes a plurality of sensor sets including a plurality of sensors, and which further comprises:

acquiring a difference among a plurality of signals detected at the plurality of sensors included in each of the sensor sets as a facial electromyography signal of each of the sensor sets.

9. The controlling method of an electronic device of claim 7 , further comprising

filtering the facial electromyography signal based on a band pass filter in a band of 20 Hz to 450 Hz.

10. The controlling method of an electronic device of claim 7 ,

wherein the extracting a feature comprises:

extracting the feature while moving a window in a predetermined size at a predetermined time interval in the facial electromyography signal.

11. The controlling method of an electronic device of claim 7 ,

wherein the feature includes at least one of a root mean square (RMS), a cepstral coefficient (CC), sample entropy (SE), or a length of a signal (WL).

12. An avatar facial expression system comprising:

a sensor device that is attached in the facial part of a user and detects a facial electromyography signal according to a facial expression, and transmits the detected facial electromyography signal to a server;

a server that extracts a feature based on the received facial electromyography signal, determines a data value of an avatar's facial movement corresponding to the extracted feature, and transmits the determined data value of an avatar's facial movement to a terminal device; and

a terminal device that controls a facial movement of the avatar based on the received data value of the avatar's facial movement and displays a facial expression of the avatar corresponding to the facial expression of the user,

wherein the server is configured to:

partition features corresponding to predetermined facial expression data into a plurality of groups based on similarity, and determine a data value of the avatar's facial movement based on a learning model having the highest similarity to a feature corresponding to the user's predetermined facial expression among a plurality of learning models trained with respect to each of the plurality of partitioned groups,

wherein the server is configured to:

calculate a distance between a center value of a group including a feature corresponding to predetermined facial expression data included in each of the plurality of learning models and a feature corresponding to the user's predetermined facial expression, and determine a learning model including a group for which the distance is the closest as the learning model having the highest similarity.