Sound-based attentive state assessment
Various implementations disclosed herein include devices, systems, and methods that determine an attentive state of a user based on physiological data associated with a physiological response of the user to an auditory stimulus. For example, an example process may include selecting an auditory stimulus based on a characteristic of an environment, presenting the auditory stimulus to a user, obtaining, using a sensor, first physiological data associated with a physiological response of the user to the auditory stimulus, and assessing an attentive state of the user based on the physiological response of the user to the auditory stimulus.
1 . A method of attentive state assessment, the method comprising:
at a device comprising a processor and a speaker:
determining a type of an environment of a user based on sensor data;
selecting an auditory stimulus based on selecting a sound from a plurality of sounds corresponding to the determined type of the environment, the auditory stimulus being selected to evoke a physiological response for assessing an attentive state of the user and is associated with a location in a three-dimensional (3D) coordinate system corresponding to the environment;
presenting the auditory stimulus to the user via the speaker at the location in the 3D coordinate system;
obtaining, using a sensor, first physiological data associated with a physiological response of the user to the auditory stimulus; and
assessing the attentive state of the user based on the physiological response of the user to the auditory stimulus.
2 . The method of claim 1 , wherein selecting the auditory stimulus comprises:
classifying one or more objects in the environment; and
selecting the auditory stimulus based on the classified one or more objects.
3 . The method of claim 1 , wherein selecting the auditory stimulus comprises:
determining, from an auditory stimuli database, one or more auditory stimuli that evoke a response for assessing attention of a user; and
selecting the auditory stimulus from the one or more auditory stimuli based on the environment.
4 . The method of claim 1 , wherein the environment is a physical environment around the user.
5 . The method of claim 1 , wherein the environment is an extended reality (XR) experience that is presented to the user.
6 . The method of claim 1 , wherein the auditory stimulus is a discrete sound, a series of sounds, or a spatialized sound.
7 . The method of claim 1 , wherein obtaining the first physiological data associated with a physiological response of the user to the auditory stimulus comprises monitoring for a response or lack of response occurring within a predetermined time following the presenting of the auditory stimulus.
8 . The method of claim 1 , wherein the first physiological data comprises electroencephalogram (EEG) amplitude data that is associated with the user.
9 . The method of claim 1 , wherein the first physiological data comprises pupillary movement that is associated with the user.
10 . The method of claim 1 , wherein the attentive state is assessed using a statistical or machine learning-based classification technique.
11 . The method of claim 1 , further comprising providing a notification to the user based on the attentive state.
12 . The method of claim 1 , further comprising identifying a portion of content associated with presenting the auditory stimulus to the user.
13 . The method of claim 1 , further comprising customizing content to be presented to the user based on the attentive state of the user.
14 . The method of claim 1 , further comprising aggregating attentive states determined for multiple users viewing content to provide feedback regarding the content.
15 . The method of claim 1 , wherein the device is a head-mounted device (HMD) and the environment comprises an extended reality (XR) environment.
16 . The method of claim 1 , wherein the auditory stimulus being selected to evoke the physiological response for assessing the attentive state of the user is based on distinguishing between an intended focused attention state and a mind wandering state.
17 . The method of claim 1 , wherein the auditory stimulus is associated with a location in a three-dimensional (3D) coordinate system corresponding to the environment, and wherein the auditory stimulus is presented to a user at the location in the 3D coordinate system.
18 . The method of claim 1 , wherein the auditory stimulus is associated with a virtual location in a three-dimensional (3D) coordinate system.
19 . A device comprising:
a speaker;
a non-transitory computer-readable storage medium; and
one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising:
determining a type of an environment of a user based on sensor data;
selecting an auditory stimulus based on selecting a sound from a plurality of sounds corresponding to the determined type of the environment, the auditory stimulus being selected to evoke a physiological response for assessing an attentive state of the user and is associated with a location in a three-dimensional (3D) coordinate system corresponding to the environment;
presenting the auditory stimulus to the user via the speaker at the location in the 3D coordinate system;
obtaining, using a sensor, first physiological data associated with a physiological response of the user to the auditory stimulus; and
assessing the attentive state of the user based on the physiological response of the user to the auditory stimulus.
20 . A non-transitory computer-readable storage medium, storing program instructions executable on a device to perform operations comprising:
determining a type of an environment of a user based on sensor data;
selecting an auditory stimulus based on a selecting a sound from a plurality of sounds corresponding to the determined type of the environment, the auditory stimulus being selected to evoke a physiological response for assessing an attentive state of the user and is associated with a location in a three-dimensional (3D) coordinate system corresponding to the environment;
presenting the auditory stimulus to the user via a speaker at the location in the 3D coordinate system;
obtaining, using a sensor, first physiological data associated with a physiological response of the user to the auditory stimulus; and
assessing the attentive state of the user based on the physiological response of the user to the auditory stimulus.
21 . The method of claim 1 , wherein selecting the auditory stimulus comprises:
classifying the environment into an environment type; and
selecting the auditory stimulus based on the environment type.