User comfort monitoring and notification
Some implementations disclosed herein present multimedia content to a user, identify a comfort state of the user (e.g., happy, unhappy, distressed, etc.) based on body data of the user (e.g., facial expressions, hand movements, physiological data, etc.), and update the multimedia content based on the identified comfort state.
1 . A method comprising:
at a device comprising a processor and a display;
obtaining data associated with a comfort state of a user while presenting a view of computer-generated reality (CGR) multimedia environment on the display during a first period of time, the comfort state determined based on sensor data associated with a body of the user;
determining that the data associated with the comfort state is a particular comfort state; and
in response to determining that the data associated with the comfort state of the user is the particular comfort state, updating the view of the CGR multimedia environment on the display for a second period of time by providing a feedback mechanism, wherein the feedback mechanism is selected based on the particular comfort state and content associated with the view of the CGR multimedia environment on the display.
2 . The method of claim 1 , wherein the feedback mechanism associated with updating the multimedia environment comprises displaying a message that provides an awareness of the particular comfort state.
3 . The method of claim 1 , wherein the feedback mechanism associated with updating the multimedia environment comprises displaying a notification to take a break from the multimedia environment.
4 . The method of claim 1 , wherein the feedback mechanism associated with updating the multimedia environment comprises providing an option to block content or block a user from the multimedia environment.
5 . The method of claim 1 , wherein the feedback mechanism associated with updating the multimedia environment comprises providing an option to report content or report another user in the multimedia environment to an authority, guardian, or moderator.
6 . The method of claim 1 , wherein the feedback mechanism associated with updating the multimedia environment comprises updating another multimedia environment of another user.
7 . The method of claim 1 , wherein the feedback mechanism associated with updating the multimedia environment comprises updating a multimedia representation of another user.
8 . The method of claim 1 , wherein determining that the data associated with the comfort state is the particular comfort state is based on classifying the comfort state as a particular comfort state from amongst a plurality of predefined classes of comfort states.
9 . The method of claim 1 , wherein determining that the data associated with the comfort state is the particular comfort state is based on a context selected from a group consisting of a setting of a physical environment or the multimedia environment, a sensitivity, a history of comfort states, and an activity.
10 . The method of claim 1 , wherein determining that the data associated with the comfort state is the particular comfort state is based on at least one of a tone of voice, a spoken language, an utterance, or breathing.
11 . The method of claim 1 , wherein the sensor data for determining the comfort state comprises at least two of:
an image captured by a camera sensor;
a motion captured by a motion sensor;
a heart rate captured by a heart rate sensor;
a blood oxygen level captured by a pulse oximeter sensor;
a blood pressure captured by a blood pressure sensor;
a temperature captured by a thermometer sensor; and
an electro-cardiogram (EKG) captured by an EKG sensor.
12 . The method of claim 1 , wherein determining that the data associated with the comfort state is the particular comfort state is based on a machine learning algorithm.
13 . The method of claim 1 , further comprising:
recording information associated with the data regarding the body, the comfort state, or the updating of the multimedia environment.
14 . The method of claim 1 , wherein the data regarding the body and the data associated with the comfort state is stored in a secure enclave on the device to restrict access and transmission to other devices.
15 . The method of claim 1 , wherein the device is a head-mounted device (HMD) and the multimedia environment comprises a computer-generated reality (CGR) environment.
16 . A device comprising:
a display;
a non-transitory computer-readable storage medium; and
one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising:
obtaining data associated with a comfort state of a user while presenting a view of computer-generated reality (CGR) multimedia environment on the display during a first period of time, the comfort state determined based on sensor data associated with a body of the user;
determining that the data associated with the comfort state is a particular comfort state; and
in response to determining that the data associated with the comfort state of the user is the particular comfort state, updating the view of the CGR multimedia environment on the display for a second period of time by providing a feedback mechanism, wherein the feedback mechanism is selected based on the particular comfort state and content associated with the view of the CGR multimedia environment on the display.
17 . The device of claim 16 , wherein the feedback mechanism associated with updating the multimedia environment comprises at least one of: i) displaying a message that provides an awareness of the particular comfort state, and ii) displaying a notification to take a break from the multimedia environment.
18 . The device of claim 16 , wherein the feedback mechanism associated with updating the multimedia environment comprises at least one of: i) providing an option to block content or block a user from the multimedia environment; and ii) providing an option to report content or report another user in the multimedia environment to an authority, guardian, or moderator.
19 . The device of claim 16 , wherein the feedback mechanism associated with updating the multimedia environment comprises at least one of: i) updating another multimedia environment of another user, and ii) updating a multimedia representation of another user.
20 . A non-transitory computer-readable storage medium, storing program instructions executable via one or more processors to perform operations comprising:
obtaining data associated with a comfort state of a user while presenting a view of computer-generated reality (CGR) multimedia environment on a display during a first period of time, the comfort state determined based on sensor data associated with a body of the user;
determining that the data associated with the comfort state is a particular comfort state; and
in response to determining that the data associated with the comfort state of the user is the particular comfort state, updating the view of the CGR multimedia environment on the display for a second period of time by providing a feedback mechanism, wherein the feedback mechanism is selected based on the particular comfort state and content associated with the view of the CGR multimedia environment on the display.