Apparatus, method and computer program for providing information relating to a cognitive state of an individual
In one embodiment there is disclosed an apparatus ( 1 ) for determining a cognitive state of a user of a mobile device ( 2 ), the apparatus ( 1 ) comprising: the mobile device ( 2 ), an IMU ( 6 ) in the mobile device ( 2 ), optical output means ( 5.1 ) in the mobile device ( 2 ), for optically giving out information to the user, a camera ( 3 ) in the mobile device ( 2 ), for recording images of an environment of the mobile device ( 2 ), an augmented reality module ( 14 ), for generating an augmented reality environment (AR) shown via the optical output means ( 5.1 ) and based on the environment (PE) of the mobile device ( 2 ) captured by the camera ( 3 ), characterised in that the apparatus ( 1 ) further comprises an enabling module ( 15 ), for enabling an input element ( 10.1 ) in the mobile device ( 2 ) once the augmented reality environment is shown, said enabling depending on data measured by the IMU ( 6 ), said input element ( 10.1 ) allowing the user to solve a task once it is selected by the user after being enabled a recording module ( 16 ), for recording data from the IMU ( 6 ) in a period of time between the input element ( 10.1 ) enablement and selection, a feature extraction module ( 8 ), for reducing the data recorded by the recording module to a set of magnitudes at given frequencies, a machine learning module ( 9 ), for determining the cognitive state of the user based on said set of magnitudes at given frequencies.
1 . Apparatus for providing information relating to a cognitive state of a first individual, the apparatus comprising:
a measurement unit operable to measure accelerometer data related to the first individual's hand movement,
an augmented reality module, for generating an augmented reality environment shown to the first individual via an optical output,
a recording module, for recording the accelerometer data from the measurement unit in a first period of time while the first individual executes at least one task in the augmented reality environment, the apparatus configured to begin the first period of time when an acceleration measured by the measurement unit is below a threshold,
a feature extraction module, for reducing the accelerometer data recorded by the recording module to a set of magnitudes at given frequencies,
a machine learning module, for providing information relating to the cognitive state of the first individual based on said set of magnitudes at given frequencies.
2 . The apparatus of claim 1 , wherein the measurement unit comprises an inertial measurement unit (IMU).
3 . The apparatus of claim 1 , wherein the first period of time is between 1.20 seconds and 1.40 seconds.
4 . The apparatus of claim 1 , wherein said set of magnitudes at given frequencies comprises N frequencies, wherein N is an integer number between 7 and 15.
5 . The apparatus of claim 1 , wherein said given frequencies are all lower than 10 Hz.
6 . The apparatus of claim 1 , wherein the frequency resolution of said set of magnitudes at given frequencies is the inverse of said first period of time.
7 . The apparatus of claim 1 , wherein said given frequencies are 0.78, 1.56, 2.34, 3.12, 3.91, 4.69, 5.47, 6.25, 7.03, 7.81.
8 . The apparatus of claim 1 comprising an input element operable to start the first period of time.
9 . The apparatus of claim 8 comprising:
a mobile device, the measurement unit being an inertial measurement unit (IMU) in the mobile device,
a camera in the mobile device, for recording images of an environment of the mobile device, wherein the augmented reality module generates the augmented reality environment based on the environment of the mobile device captured by the camera,
an enabling module, for enabling the input element once the augmented reality environment is shown, said enabling depending on the accelerometer data measured by the IMU, said input element allowing the first individual execute the at least one task once it is selected by the first individual after being enabled,
wherein the first period of time is a period of time between the input element enablement and selection.
10 . The apparatus of claim 9 , the enabling module being arranged to enable the input element depending on at least one image taken by the camera.
11 . The apparatus of claim 9 , the enabling module being arranged to enable the input element depending on the presence of a determined element in at least one image taken by the camera.
12 . The apparatus of claim 9 , comprising a path module arranged for determining a positional path of the mobile device in the environment, wherein the machine learning module is arranged for determining the cognitive state of the user based also on the positional path of the mobile device.
13 . The apparatus of claim 9 , wherein the augmented reality module is arranged to augment the visualization of the surrounding using augmented reality glasses in combination with the mobile device once the user is at a predetermined distance.
14 . A method for providing information relating to a cognitive state of a first individual, by using an apparatus comprising:
a measurement unit operable to measure accelerometer data related to the first individual's hand movement, wherein the method comprises the following steps
generating an augmented reality environment shown to the first individual via an optical output,
recording accelerometer data from the measurement unit in a first period of time, while the first individual executes at least one task in the augmented reality environment, by using a recording module of said apparatus, the first period of time beginning when an acceleration measured by the measurement unit is below a threshold
reducing the accelerometer data recorded by the recording module to a set of magnitudes at given frequencies, by using a feature extraction module of said apparatus,
providing information relating to the cognitive state of the first individual based on said set of magnitudes at given frequencies, by using a machine learning module of said apparatus.
15 . The method of claim 14 , further comprising starting the first period of time using an input element in the apparatus.
16 . The method of claim 15 , wherein the apparatus comprises a mobile device and the measurement unit comprises an inertial measurement unit (IMU) in the mobile device, the method further comprising:
recording images of an environment of the mobile device using a camera in the mobile device;
generating the augmented reality environment using the augmented reality module based on the environment of the mobile device captured by the camera; and
enabling the input element using an enablement module once the augmented reality environment is shown, said enabling depending on the accelerometer data measured by the IMU, said input element allowing the first individual to execute the at least one task once it is selected by the first individual after being enabled,
wherein the first period of time is a period of time between the input element enablement and selection.
17 . Apparatus for providing information relating to a cognitive state of a first individual, the apparatus comprising:
a measurement unit operable to measure accelerometer data related to the first individual's hand movement,
an augmented reality module, for generating an augmented reality environment shown to the first individual via an optical output,
a recording module, for recording the accelerometer data from the measurement unit in a first period of time while the first individual executes at least one task in the augmented reality environment,
a feature extraction module, for reducing the accelerometer data recorded by the recording module to a set of magnitudes at given frequencies, wherein said given frequencies are all lower than 10 Hz,
a machine learning module, for providing information relating to the cognitive state of the first individual based on said set of magnitudes at given frequencies.