Method and electronic device for predicting emotion of user

1 . A method of operating an electronic device, the method comprising:

receiving, by the electronic device, a user context, a device context and an environment context, wherein the user context, the device context, and the environment context are collected by at least one of the electronic device and at least one of one or more other electronic devices connected to the electronic device;

determining, by the electronic device, a combined representation of the user context, the device context and the environment context;

determining, by the electronic device, a plurality of user characteristics based on the combined representation of the user context, the device context and the environment context;

predicting, by the electronic device, an emotion of a user based on the plurality of user characteristics and the combined representation of the user context, the device context, the environment context; and

personalizing, by the electronic device, content on the electronic device and on at least one of the one or more other electronic devices based on the predicted emotion of the user,

wherein the personalizing the content comprises at least one of:

adjusting a speed of an animation on a display of the electronic device;

adjusting a duration of a security lock on the display of the electronic device; or

adjusting a color palette of a user interface of the electronic device, and

wherein the determining, by the electronic device, the combined representation of the user context, the device context and the environment context comprises:

determining a plurality of features associated with the user from the user context, the device context and the environment context;

segregating the plurality of features associated with the user into a plurality of categories corresponding to a specific duration of time;

generating, using encoding, at least one vector representation for each of the plurality of categories; and

determining the combined representation of the user context, the device context and the environment context based on the at least one vector representation for each of the plurality of categories.

2 . The method of claim 1 , further comprising:

performing, by the electronic device, based on the predicted emotion of the user, at least one of:

modifying a user experience on the electronic device and on at least one of the one or more other electronic devices,

utilizing an emotional profile on the electronic device and on at least one of the one or more other electronic devices,

generating at least one object for providing an emotional support to the user,

providing a security function to the user in a virtual environment, and

modifying at least one user parameter in the virtual environment.

3 . The method of claim 1 , further comprising:

determining, by the electronic device, at least one of: a consumption of content by the user, abnormal usage pattern on the electronic device or on at least one of the one or more other electronic devices, a recurrence activity performed on the electronic device or on at least one of the one or more other electronic devices by the user, and a time duration spent by the user on the electronic device or on at least one of the one or more other electronic devices; and

determining, by the electronic device, a quality of the predicted emotion of the user, wherein the quality of the predicted emotion is a positive emotion or a negative emotion.

4 . The method of claim 1 , wherein the determining, by the electronic device, the plurality of user characteristics based on the combined representation of the user context, the device context and the environment context comprises:

providing, by the electronic device, the combined representation of the user context, the device context and the environment context to a first network and a plurality of intermediate models; and

determining, by the electronic device, the plurality of user characteristics.

5 . The method of claim 4 , further comprising:

predicting, by the electronic device, a first set of intermediate emotions based on the plurality of user characteristics and the combined representation of the user context, the device context and the environment context.

6 . The method of claim 4 , further comprising:

providing, by the electronic device, the combined representation of the user context, the device context and the environment context to a second network and a third network;

determining, by the electronic device, a local graph emotion prediction from the second network and a global node prediction from the third network;

combining, by the electronic device, the local graph emotion prediction and the global node prediction based on a specific weight; and

predicting, by the electronic device, a second set of intermediate emotions.

7 . The method of claim 1 , wherein the predicting, by the electronic device, the emotion of the user based on the combined representation of the user context, the device context, the environment context and the plurality of user characteristics comprises:

receiving, by at least one second model of the electronic device, a first set of intermediate emotions and a second set of intermediate emotions;

receiving, by the at least one second model of the electronic device, a categorical clustering map;

performing, by the at least one second model of the electronic device, an ensembling technique on the first set of intermediate emotions and the second set of intermediate emotions based on the categorical clustering map; and

predicting, by the electronic device, the emotion of the user.

8 . The method of claim 1 , wherein the plurality of user characteristics is determined using at least one first model and wherein the emotion of the user is predicted using at least one second model.

9 . An electronic device comprising:

at least one memory configured to store at least one instruction;

at least one processor in communication with the at least one memory and configured to execute the at least one instruction; and

a communicator in communication with the at least one memory and the at least one processor,

wherein the at least one instruction, when executed by the at least one processor, causes the electronic device to:

receive a user context, a device context and an environment context, wherein the user context, the device context, and the environment context are collected by at least one of the electronic device and at least one of one or more other electronic devices connected to the electronic device;

determine a combined representation of the user context, the device context and the environment context by:

determining a plurality of features associated with a user from the user context, the device context and the environment context,

segregating the plurality of features associated with the user into a plurality of categories corresponding to a specific duration of time,

generating at least one vector representation for each of the plurality of categories, and

determining the combined representation of the user context, the device context and the environment context based on the at least one vector representation for each of the plurality of categories;

determine a plurality of user characteristics based on the combined representation of the user context, the device context and the environment context;

predict an emotion of the user based on the plurality of user characteristics and the combined representation of the user context, the device context, the environment context; and

personalize content on the electronic device and on at least one of the one or more other electronic devices based on the predicted emotion of the user by performing at least one of:

adjusting a speed of an animation on a display of the electronic device,

adjusting a duration of a security lock on the display of the electronic device, or

adjusting a color palette of a user interface of the electronic device.

10 . The electronic device of claim 9 , wherein the at least one instruction, when executed by the at least one processor, further causes the electronic device to perform, based on the predicted emotion of the user, at least one of:

modifying a user experience on the electronic device and on at least one of the one or more other electronic devices,

utilizing an emotional profile on the electronic device and on at least one of the one or more other electronic devices,

generating at least one object for providing an emotional support to the user,

providing a security function to the user in a virtual environment, and

modifying at least one user parameter in the virtual environment.

11 . The electronic device of claim 9 , wherein the at least one instruction, when executed by the at least one processor, further causes the electronic device to:

determine at least one of: a consumption of content by the user, abnormal usage pattern on the electronic device or on at least one of the one or more other electronic devices, a recurrence activity performed on the electronic device or on at least one of the one or more other electronic devices by the user, and a time duration spent by the user on the electronic device or on at least one of the one or more other electronic devices; and

determine a quality of the predicted emotion of the user, wherein the quality of the predicted emotion is a positive emotion or a negative emotion.

12 . The electronic device of claim 9 , wherein the at least one instruction, when executed by the at least one processor, further causes the electronic device to:

determine the plurality of user characteristics based on the combined representation of the user context, the device context and the environment context by providing the combined representation of the user context, the device context and the environment context to a first network and a plurality of intermediate models.

13 . The electronic device of claim 12 , wherein the at least one instruction, when executed by the at least one processor, further causes the electronic device to:

predict a first set of intermediate emotions based on the plurality of user characteristics and the combined representation of the user context, the device context and the environment context.

14 . The electronic device of claim 12 , wherein the at least one instruction, when executed by the at least one processor, further causes the electronic device to:

provide the combined representation of the user context, the device context and the environment context to a second network and a third network;

determine a local graph emotion prediction from the second network and a global node prediction from the third network;

combine the local graph emotion prediction and the global node prediction based on a specific weight; and

predict a second set of intermediate emotions.

15 . The electronic device of claim 9 , wherein the at least one instruction, when executed by the at least one processor, further causes the electronic device to predict the emotion of the user based on the plurality of user characteristics and the combined representation of the user context, the device context, the environment context by:

receiving, by at least one second model of the electronic device, a first set of intermediate emotions and a second set of intermediate emotions;

receiving, by the at least one second model of the electronic device, a categorical clustering map;

performing, by the at least one second model of the electronic device, an ensembling technique on the first set of intermediate emotions and the second set of intermediate emotions based on the categorical clustering map; and

predicting, by the electronic device, the emotion of the user.

16 . The electronic device of claim 9 , wherein the at least one instruction, when executed by the at least one processor, further causes the electronic device to:

determine a plurality of user characteristics based on the combined representation of the user context, the device context and the environment context using at least one first model, and

predict the emotion of the user using at least one second model.