Systems and methods for reaction measurement
This disclosure relates to systems, media, and methods for quantifying and monitoring visual acuity, motor skill, and cognitive ability of a user through measuring user activities while the user responds to prompts on an electronic device. Disclosed embodiments may receive image data from a user-facing camera, motion data from a sensor device attached to the user, location data from a touchscreen, and time data measuring a user's response times. Disclosed embodiments may provide graphical user interfaces to which the user responds and calculate health metrics based on characteristics of the user's response. Disclosed embodiments may calculate visual acuity, motor skill, and cognitive ability metrics based on data measured while the user responds to the graphical user interfaces. Disclosed embodiments may include comparing the metrics to respective baseline values and providing an alert when the metrics deviate from threshold values indicating a change in user health.
1. A system for measuring reaction times of a user, comprising:
an electronic device comprising a touchscreen, a user-facing camera, at least one memory storing operation instructions, and at least one processor;
a first sensor device attached to the user, the first sensor device comprising an inertial measurement unit;
the at least one processor executing the operation instructions to perform operations comprising:
providing, on the touchscreen, a graphical user interface with at least one selection region;
updating, at an interval, the graphical user interface to provide at least one object related to the at least one selection region;
receiving a user selection on the graphical user interface of the touchscreen;
recording, between the updating and the receiving, data comprising:
image data, from the user-facing camera, including at least a portion of a face of the user;
first motion data, from the first sensor device, representing first motions of the user;
location data, from the touchscreen, representing a location of the user selection within the graphical user interface; and
time data, indicating a time period between the updating and the receiving;
determining, based on the location data, if the user performed reaction test instructions comprising selecting one of the at least one selection region on the touchscreen when at least one corresponding object appears;
calculating, based on the image data, a visual acuity metric as an average percent of a field of view of the camera that includes the user's face;
calculating, based on the first motion data, a motor skill metric as an average amount of time the user moves between the updating and the receiving;
calculating, based on the time data, a cognitive ability metric as an average time period between the updating and the receiving,
comparing the visual acuity metric, the motor skill metric, and the cognitive ability metric to a visual acuity threshold, a motor skill threshold, and a cognitive ability threshold, respectively; and
providing, based on the comparing, an alert based on at least one of the following occurring:
the visual acuity metric exceeding the visual acuity threshold,
the motor skill metric exceeding the motor skill threshold, or
the cognitive ability metric exceeding the cognitive ability threshold.
2. The system of claim 1 , wherein the operations further comprise:
storing the visual acuity metric, the motor skill metric, and the cognitive ability metric;
setting a baseline value of visual acuity, a baseline value of motor skill, and a baseline value of cognitive ability as an average of the stored visual acuity metrics, stored motor skill metrics, and stored cognitive ability metrics, respectively; and
setting the visual acuity threshold, the motor skill threshold, and the cognitive ability thresholds based on respective baseline values.
3. The system of claim 1 , wherein the operations further comprise waiting for a predefined period before updating the graphical user interface.
4. The system of claim 3 , wherein the operations further comprise:
providing a user-entry field;
receiving a user entry in the user-entry field; and
setting the predefined period to the user entry.
5. The system of claim 1 , wherein the operations further comprise waiting for a random period before updating the graphical user interface, wherein the random period is within a predefined range.
6. The system of claim 1 , wherein the operations further comprise:
updating the graphical user interface to display the object in a size and a color that differ from a size and color of the object prior to the update; and
calculating the visual acuity metric based on the size or color of the objects.
7. The system of claim 1 , wherein the operations further comprise:
calculating the motor skill metric based on a number of accelerations in the first motion data that exceed an acceleration threshold.
8. The system of claim 1 , wherein the operations further comprise:
updating the graphical user interface to provide a display corresponding to reaction test instructions having a difficulty level, based on a number of objects displayed, that differs from a difficulty level of a previous iteration; and
calculating the cognitive ability metric based on the difficulty level of each iteration.
9. The system of claim 1 , wherein the operations further comprise:
identifying locations of the user's pupils in the image data;
calculating, based on the user's pupil locations, where the user looks;
calculating, based on where the user looks, an amount of time the user spends looking at the touchscreen; and
calculating the cognitive ability metric based on the amount of time the user spends looking at the touchscreen.
10. The system of claim 1 , wherein the operations further comprise:
asking if the user is currently wearing eyeglasses;
receiving a user eyeglasses response;
identifying the user's eyeglasses in the image data;
calculating, based on the user's eyeglasses location, where the user looks;
calculating, based on where the user looks, an amount of time the user spends looking at the touchscreen; and
calculating the cognitive ability metric based on the amount of time the user spends looking at the touchscreen.
11. The system of claim 1 , further comprising:
a second sensor device attached to the user, the second sensor device comprising an inertial measurement unit, the first sensor device being worn on a first hand of the user and the second sensor device being worn on a second hand of the user;
the reaction test instructions further comprise specifying which hand the user must use when selecting the one of the at least one selection region; and
the operations further comprise:
recording, between the updating and the receiving, second motion data, from the second sensor device, representing a second motion of the user;
determining if the user performed the reaction test instructions with the correct hand based on the first motion data and second motion data;
calculating, based on the first motion data, a first hand motor skill metric as an average amount of time the user moves between the updating and the receiving when the reaction test instructions specify that the first hand must be used; and
calculating, based on the second motion data, a second hand motor skill metric as an average amount of time the user moves between the updating and the receiving when the reaction test instructions specify that the second hand must be used.
12. The system of claim 1 , wherein:
the reaction test instructions comprise:
selecting a first selection region when a first object appears; and
selecting a second selection region when a second object appears; and
the operations further comprise:
providing the graphical user interface with the first selection region and the second selection region;
randomly selecting the first object or the second object;
updating the graphical user interface to provide the randomly selected object;
determining, based on the location data, if the user selected the selection region corresponding to the randomly selected object;
calculating an average correct response time as the average of time periods where the user selected the selection region corresponding to the randomly selected object;
calculating an average incorrect response time as the average, multiplied by a penalty factor, of time periods where the user did not select the selection region corresponding to the randomly selected object; and
calculating the cognitive ability metric as an average of the average correct response time and the average incorrect response time.
13. The system of claim 1 , wherein:
the reaction test instructions comprise:
selecting a first selection region when a first object appears on a first side of the touchscreen; and
selecting a second selection region when the first object appears on a second side of the touchscreen;
the operations further comprise:
providing the graphical user interface with the first selection region and the second selection region;
randomly selecting the first side or the second side;
updating the graphical user interface to provide the first object on the randomly selected side and a second object on a remaining side;
determining, based on the location data, if the user selected the selection region corresponding to the randomly selected side;
calculating an average correct response time as the average of time periods where the user selected the selection region corresponding to the randomly selected side;
calculating an average incorrect response time as the average, multiplied by a penalty factor, of time periods where the user did not select the selection region corresponding to the randomly selected side; and
calculating the cognitive ability metric as an average of the average correct response time and the average incorrect response time.
14. The system of claim 1 , wherein:
the reaction test instructions comprise:
selecting the one of the at least one selection region when a first object appears; and
not selecting the one of the at least one selection region when a second object appears; and
the operations further comprise:
randomly selecting the first object or the second object;
updating the graphical user interface to provide the selected object;
updating the graphical user interface if a time limit expires and the user did not select the one of the at least one selection region.
15. The system of claim 1 , wherein:
the reaction test instructions comprise:
selecting the one of the at least one selection region when a number of first objects exceeds a number of second objects; and
not selecting the one of the at least one selection region when the number of first objects is less than the number of second objects; and
the operations further comprise:
randomly selecting the number of first objects and the number of second objects to display on the interface; and
updating the graphical user interface to provide the first objects and the second objects; and
updating the graphical user interface if a time limit expires and the user did not select the one of the at least one selection region.
16. The system of claim 1 , wherein the operations further comprise:
updating the graphical user interface to display a test complete message if the graphical user interface has been updated a predefined number of times.
17. The system of claim 1 , wherein the operations further comprise:
updating the graphical user interface to provide an incomplete test screen if a time limit has expired and an exit button is selected, the incomplete test screen comprising a save button; and
storing the visual acuity metric, the motor skill metric, and the cognitive ability metric if the save button is selected.
18. The system of claim 1 , wherein the operations further comprise:
calculating an average of the time data;
calculating, based on the determining, a correct selection rate as a ratio of a number of correct selections to a number of incorrect selections;
calculating a reaction metric as a quotient of the average time data and the correct selection rate;
calculating, based on the first motion data, a number of accelerations above a threshold;
calculating, based on the image data, a number of times the user looks away from the touchscreen; and
calculating a total score as a sum of the reaction metric, the number of accelerations above the threshold, and the number of times the user looks away.
19. The system of claim 1 , wherein
the electronic device further comprises a touchscreen motion sensor;
the reaction test instructions comprise a direction to move the electronic device; and
the operations further comprise:
recording, between the updating and the receiving, data comprising second motion data, from the touchscreen motion sensor, representing second motions of the user;
determining if the user correctly performed the reaction test instructions based on the second motion data; and
calculating the motor skill metric on the basis of a number of accelerations, measured by the touchscreen motion sensor, that exceed an acceleration threshold.
20. The system of claim 1 , wherein:
the reaction test instructions comprise a direction that the user must touch a physical item before selecting the one of the at least one selection region; and
the operations further comprise:
determining, based on a number of accelerations of the first motion data, if the user correctly performed the reaction test instructions; and
calculating the motor skill metric based on a number of accelerations above a first acceleration threshold and below a second acceleration threshold.
21. A method for measuring reaction times of a user, comprising:
providing an electronic device comprising a touchscreen and a user-facing camera;
providing a first sensor device attached to the user, the first sensor device comprising an inertial measurement unit;
providing, on the touchscreen, a graphical user interface with at least one selection region;
updating, at an interval, the graphical user interface to provide at least one object related to the at least one selection region;
receiving a user selection on the graphical user interface of the touchscreen;
recording, between the updating and the receiving, data comprising:
image data, from the user-facing camera, including at least a portion of a face of the user;
first motion data, from the first sensor device, representing first motions of the user;
location data, from the touchscreen, representing a location of the user selection within the graphical user interface; and
time data, indicating a time period between the updating and the receiving;
determining, based on the location data, if the user performed reaction test instructions comprising selecting one of the at least one selection region on the touchscreen when at least one corresponding object appears;
calculating, based on the image data, a visual acuity metric as an average percent of a field of view of the camera that includes the user's face;
calculating, based on the first motion data, a motor skill metric as an average amount of time the user moves between the updating and the receiving;
calculating, based on the time data, a cognitive ability metric as an average time period between the updating and the receiving,
comparing the visual acuity metric, the motor skill metric, and the cognitive ability metric to a visual acuity threshold, a motor skill threshold, and a cognitive ability threshold, respectively; and
providing, based on the comparing, an alert based on at least one of the following occurring:
the visual acuity metric exceeding the visual acuity threshold,
the motor skill metric exceeding the motor skill threshold, or
the cognitive ability metric exceeding the cognitive ability threshold.
22. A non-transitory computer-readable media storing instructions that, when executed on one or more processors, cause the one or more processors to perform operations for measuring reaction times of a user, comprising:
providing, on a touchscreen of an electronic device, a graphical user interface with at least one selection region;
updating, at an interval, the graphical user interface to provide at least one object related to the at least one selection region;
receiving a user selection on the graphical user interface of the touchscreen;
recording, between the updating and the receiving, data comprising:
image data, from a user-facing camera, including at least a portion of a face of the user;
first motion data, from a first sensor device comprising an inertial measurement unit attached to the user, representing first motions of the user;
location data, from the touchscreen, representing a location of the user selection within the graphical user interface; and
time data, indicating a time period between the updating and the receiving;
determining, based on the location data, if the user performed reaction test instructions comprising selecting one of the at least one selection region on the touchscreen when at least one corresponding object appears;
calculating, based on the image data, a visual acuity metric as an average percent of a field of view of the camera that includes the user's face;
calculating, based on the first motion data, a motor skill metric as an average amount of time the user moves between the updating and the receiving;
calculating, based on the time data, a cognitive ability metric as an average time period between the updating and the receiving,
comparing the visual acuity metric, the motor skill metric, and the cognitive ability metric to a visual acuity threshold, a motor skill threshold, and a cognitive ability threshold, respectively; and
providing, based on the comparing, an alert based on at least one of the following occurring:
the visual acuity metric exceeding the visual acuity threshold,
the motor skill metric exceeding the motor skill threshold, or
the cognitive ability metric exceeding the cognitive ability threshold.