Information processing device and control method

1. An information processing device, comprising:

a central processing unit (CPU) configured to:

acquire a first sensor value from a first measurement sensor and a second sensor value from a second measurement sensor, wherein

each of the first measurement sensor and the second measurement sensor is one of wearable or holdable by a first user,

the first sensor value corresponds to biological information of the first user,

the second sensor value corresponds to expression information of a second user, and

the second user is an interaction partner of the first user;

recognize, based on the biological information of the first user and the expression information of the second user, a first emotion of the second user, wherein the first emotion of the second user corresponds to an emotion of the second user at a current time;

propose a specific action to the first user based on the first emotion of the second user and user information of the second user, wherein the user information of the second user includes at least one of schedule information, positional information, or preference information of the second user;

estimate a future emotion of the second user based on an emotion history of the second user, wherein the future emotion of the second user corresponds to an emotion of the second user at a time in future;

compare the future emotion of the second user with the first emotion of the second user;

determine a change in a mood of the second user based on the comparison of the future emotion of the second user with the first emotion of the second user; and

control, based on the first emotion of the second user, concurrent display of a notification and information indicating the proposed specific action to the first user, wherein the notification to the first user includes a result of the determination of the change in the mood of the second user.

2. The information processing device according to claim 1 , wherein

the biological information of the first user includes one of a heart rate sensor value or a sweat sensor value, and

the CPU is further configured to:

extract a feature quantity from the biological information of the first user; and

recognize the first emotion of the second user based on the feature quantity extracted from the biological information of the first user.

3. The information processing device according to claim 1 , wherein the CPU is further configured to:

extract a feature quantity from the biological information of the first user;

recognize an emotion of the first user based on the feature quantity extracted from the biological information of the first user; and

recognize, as the first emotion of the second user, a second emotion that one of matches, resembles, or correlates with the emotion of the first user.

4. The information processing device according to claim 1 , wherein the first sensor value further corresponds to amount-of-activity information of the first user.

5. The information processing device according to claim 4 , wherein the amount-of-activity information includes one of an acceleration sensor value, an angular velocity sensor value, a pedometer value, or a geomagnetic sensor value.

6. The information processing device according to claim 1 , wherein

the expression information includes a position of a corner of one of a mouth or an eyebrow of the second user based on a feature point extracted from a face image of the second user, and

the CPU is further configured to:

extract a feature quantity from the expression information; and

recognize the first emotion of the second user based on the feature quantity extracted from the expression information.

7. The information processing device according to claim 1 , wherein the second sensor value further corresponds to one of collected voice information of the second user, sensed posture information of the second user, or temperature information of the second user acquired from an infrared image.

8. The information processing device according to claim 1 , wherein the CPU is further configured to recognize the first emotion of the second user based on biological information of a third party.

9. The information processing device according to claim 1 , wherein the CPU is further configured to:

accumulate information regarding the first emotion of the second user in a database; and

extract the emotion history of the second user from the database.

10. The information processing device according to claim 9 , wherein

the CPU is further configured to control display of an emotion forecast of the second user, and

the display of the emotion forecast is based on a result of the estimation of the future emotion of the second user.

11. The information processing device according to claim 9 , wherein the CPU is further configured to:

recognize the first emotion of the second user based on action-related information of the second user and the emotion history of the second user; and

control, based on the recognized first emotion of the second user, display of the notification to the first user.

12. The information processing device according to claim 9 , wherein the CPU is further configured to estimate the future emotion of the second user based on the emotion history of the second user and action-related information of the second user.

13. The information processing device according to claim 1 , wherein the notification includes a graphical representation of a transition of the first emotion of the second user in a time series manner.

14. A control method, comprising:

in an information processing device that includes a central processing unit (CPU):

acquiring, by the CPU, a first sensor value from a first measurement sensor and a second sensor value from a second measurement sensor, wherein

each of the first measurement sensor and the second measurement sensor is one of wearable or holdable by a first user,

the first sensor value corresponds to biological information of the first user,

the second sensor value corresponds to expression information of a second user, and

the second user is an interaction partner of the first user;

recognizing, by the CPU, a current emotion of the second user based on the biological information of the first user and the expression information of the second user, wherein the current emotion of the second user corresponds to an emotion of the second user at a current time;

proposing, by the CPU, a specific action to the first user based on the current emotion of the second user and user information of the second user, wherein the user information of the second user includes at least one of schedule information, positional information, or preference information of the second user;

estimating, by the CPU, a future emotion of the second user based on an emotion history of the second user, wherein the future emotion of the second user corresponds to an emotion of the second user at a time in future;

comparing, by the CPU, the future emotion of the second user with the current emotion of the second user;

determining, by the CPU, a change in a mood of the second user based on the comparison of the future emotion of the second user with the current emotion of the second user; and

controlling, by the CPU based on the emotion of the second user, concurrent display of a notification and information indicating the proposed specific action to the first user, wherein the notification to the first user includes a result of the determination of the change in the mood of the second user.

15. A non-transitory computer-readable medium having stored thereon computer-executable instructions which, when executed by a processor, cause the processor to execute operations, the operations comprising:

acquiring a first sensor value from a first measurement sensor and a second sensor value from a second measurement sensor, wherein

each of the first measurement sensor and the second measurement sensor is one of wearable or holdable by a first user,

the first sensor value corresponds to biological information of the first user,

the second sensor value corresponds to expression information of a second user, and

the second user is an interaction partner of the first user;

recognizing, based on the biological information of the first user and the expression information of the second user, a current emotion of the second user, wherein the current emotion of the second user corresponds to an emotion of the second user at a current time;

proposing a specific action to the first user based on the current emotion of the second user and user information of the second user, wherein the user information of the second user includes at least one of schedule information, positional information, or preference information of the second user;

estimating a future emotion of the second user based on an emotion history of the second user, wherein the future emotion of the second user corresponds to an emotion of the second user at a time in future;

comparing the future emotion of the second user with the current emotion of the second user;

determining a change in a mood of the second user based on the comparison of the future emotion of the second user with the current emotion of the second user; and

controlling, based on the emotion of the second user, concurrent display of a notification and information indicating the proposed specific action to the first user, wherein the notification to the first user includes a result of the determination of the change in the mood of the second user.