Assessing developmental disorders via eye tracking
Systems, devices, apparatus, methods, and computer-readable storage media for developmental assessment using eye tracking are provided. In one aspect, a system for development assessment via eye tracking includes: a patient-side computing device having a screen for presenting visual stimuli to a patient, an eye-tracking device integrated with the patient-side computing device and configured to collect eye-tracking data of the patient while the visual stimuli are presented to the patient on the screen of the patient-side computing device, and an operator-side computing device configured to present a user interface for an operator to communicate with the patient-side computing device.
1. A computer-implemented method of using an infrared eye-tracker camera of a handheld eye-tracker console that is wirelessly controlled by an operator-side handheld computer physically untethered from the handheld eye-tracker console, the method comprising:
wirelessly connecting the handheld eye-tracker console with the operator-side handheld computer via a cloud server that provides wireless communication between the operator-side handheld computer and the handheld eye-tracker console while both a display screen and the infrared eye-tracker camera of the handheld eye-tracker console are mechanically mounted together and oriented in a same direction and while the operator-side handheld computer is portable to different locations relative to the handheld eye-tracker console;
during said wireless communication between the operator-side handheld computer and the handheld eye-tracker console via the cloud server and in response to a command input at the operator-side handheld computer while a patient is oriented toward the handheld eye-tracker console, operating the infrared eye-tracker camera mounted adjacent to a periphery of the display screen of the handheld eye-tracker console to convert infrared-detected eye movements of the patient into a real-time data stream that includes pupil positions and gaze vectors during display of a predetermined sequence of stimulus videos on the display screen of the handheld eye-tracker console; and
subsequent to said operating the infrared eye-tracker camera of the handheld eye-tracker in response to the command input at the operator-side handheld computer, automatically uploading to the cloud server without human intervention both a first file storing timestamped eye-tracking gaze coordinate data of the patient collected during the display of the predetermined sequence of stimulus videos and a second file storing a playlist of the predetermined sequence of stimulus videos and associated timestamps of the predetermined sequence of stimulus videos.
2. The computer-implemented method of claim 1 , wherein the operator-side handheld computer comprises an operator-side portable tablet computing device untethered from the handheld eye-tracker console and movable to a location outside the patient's field of view.
3. The computer-implemented method of claim 2 , further comprising:
subsequent to wirelessly connecting the handheld eye-tracker console with the operator-side handheld computer via the cloud server, capturing an image of eyes of the patient using the handheld eye-tracker console while the patient is oriented toward the handheld eye-tracker console such that the captured image is indicative of a relative position between eyes of the patient and the infrared eye-tracker camera mechanically mounted adjacent to the periphery of the display screen of the handheld eye-tracker console; and
responsive to the captured image of the eyes of the patient indicating that the relative position between said eyes of the patient and the infrared eye-tracker camera of the handheld eye-tracker console is within the predetermined threshold, displaying on a user interface of the operator-side portable tablet computing device an indication that the patient is aligned with the infrared eye-tracker camera of the handheld eye-tracker console.
4. The computer-implemented method of claim 3 , wherein:
said capturing said image of the eyes of the patient using the handheld eye-tracker console is initiated in response to operator input of a button selection on a user interface of the operator-side portable tablet computing device, which causes the cloud server to send a command to the handheld eye-tracker console to capture said image of the eyes of the patient.
5. The computer-implemented method of claim 1 , wherein said wirelessly connecting the handheld eye-tracker console with the operator-side handheld computer via the cloud server comprises establishing a wireless connection between the operator-side handheld computer and the handheld eye-tracker console by:
accessing, by the operator-side handheld computer, a web portal at the cloud server; and
in response to receiving a selection of the handheld eye-tracker console at an onscreen interface of the operator-side handheld computer connected to the cloud server via the web portal, wirelessly connecting the operator-side handheld computer to the handheld eye-tracker console.
6. The computer-implemented method of claim 5 , wherein said establishing the wireless connection between the operator-side handheld computer and the handheld eye-tracker console comprises:
displaying, by the handheld eye-tracker console, an access code automatically generated by the cloud server without human intervention on the display screen of the handheld eye-tracker console; and
in response to receiving manual user input of said access code at the operator-side handheld computer, establishing the wireless connection between the operator-side handheld computer and the handheld eye-tracker console.
7. The computer-implemented method of claim 1 , further comprising:
subsequent to wirelessly connecting the handheld eye-tracker console with the operator-side handheld computer via the cloud server and prior to operating the infrared eye-tracker camera of the handheld eye-tracker console in response to the command at the operator-side handheld computer, displaying visual desensitization video on the display screen of the handheld eye-tracker console such that the desensitization video causes exogenous cueing by the patient without the need for verbal mediation or instruction.
8. The computer-implemented method of claim 7 , further comprising: controlling the infrared eye-tracker camera of the handheld eye-tracker console to cease collection of eye-tracking data of the patient while displaying the visual desensitization video on the display screen of the handheld eye-tracker console.
9. The computer-implemented method of claim 8 , further comprising: while the patient is oriented toward the visual desensitization video displayed on the display screen of the handheld eye-tracker console, receiving a selection of a patient profile for the patient at the operator-side handheld computer to set up the session for the patient.
10. The computer-implemented method of claim 9 , wherein said receiving the selection of the patient profile at the operator-side handheld computer comprises: selecting the patient among a list of patients from the cloud server that is associated with the operator of the operator-side handheld computer.
11. The computer-implemented method of claim 1 , further comprising:
subsequent to wirelessly connecting the handheld eye-tracker console with the operator-side handheld computer via the cloud server and prior to operating the infrared eye-tracker camera of the handheld eye-tracker console in response to the command at the operator-side handheld computer, displaying one or more calibration targets on the display screen of the handheld eye-tracker console to trigger a saccade towards, and the patient's fixation upon, a predetermined target location on the display screen of the handheld eye-tracker console, wherein said displaying one or more fixation targets at the handheld eye-tracker console is initiated in response to operator input of a calibration start command at the operator-side handheld computer.
12. The computer-implemented method of claim 11 , wherein said displaying one or more calibration targets comprises: sequentially presenting each of the one or more calibration targets at a corresponding predetermined target location of the display screen of the handheld eye-tracker console while capturing eye-tracking calibration data of the patient using the infrared eye-tracker camera of the handheld eye-tracker console; and determining at the handheld eye-tracker console, for each of the one or more calibration targets and based upon the captured eye-tracking calibration data, a deviation between a position of the corresponding visual fixation of the patient with the corresponding predetermined target location.
13. The computer-implemented method of claim 12 , further comprising:
in response to said determining at the handheld eye-tracker console that the deviation between the position of the corresponding visual fixation of the patient with the corresponding predetermined target location is smaller than or equal to a predetermined calibration threshold, determining that the calibration target is calibrated and displaying a next calibration target, or
in response to said determining at the handheld eye-tracker console that the deviation is greater than the predetermined calibration threshold, determining that the calibration target fails to be calibrated and re-displaying the calibration target for calibration.
14. The computer-implemented method of claim 1 , further comprising changing the display screen of the handheld eye-tracker console to present a centering target prior to presenting each stimulus video in the predetermined sequence of stimulus videos on the display screen.
15. The computer-implemented method of claim 1 , further comprising during said display of the predetermined sequence of stimulus videos on the display screen of the handheld eye-tracker console:
performing a calibration of the patient to the eye-tracker device of the handheld eye-tracker console between presenting two adjacent stimulus videos among the predetermined sequence of stimulus videos,
wherein the timestamped eye-tracking gaze coordinate data collected in performing the calibration is used for at least one of calibrating the eye-tracking data of the patient or for determining a calibration accuracy.
16. The computer-implemented method of claim 1 , further comprising:
modifying a user interface of the operator-side handheld computer while the handheld eye-tracker console displays the predetermined sequence of stimulus videos on the display screen of the handheld eye-tracker console using an onscreen graphic indicator at the operator-side handheld computer that updates in response to completion of each stimulus video in the predetermined sequence of stimulus videos displayed at the handheld eye-tracker console.
17. The computer-implemented method of claim 1 , wherein the operator-side handheld computer is configured to wirelessly communicate, via the cloud server, with said handheld eye-tracker console and multiple other handheld eye-tracker consoles, wherein said wirelessly connecting said handheld eye-tracker console with the operator-side handheld computer via the cloud server includes receiving a selection at an onscreen interface the operator-side handheld computer of said handheld eye-tracker console from among an onscreen listing of the multiple other handheld eye-tracker consoles.
18. The computer-implemented method of claim 1 , wherein the operator-side handheld computer comprises a user interface that is different from the handheld eye-tracker console.
19. The computer-implemented method of claim 18 , wherein the predetermined sequence of stimulus videos to be displayed on the handheld eye-tracker console is defined in response to the operator-side handheld computing device identifying an age of the patient.
20. The computer-implemented method of claim 1 , wherein said operating the infrared eye-tracker camera of the handheld eye-tracker in response to the command input at the operator-side handheld computer comprises activating an infrared illumination source and an infrared sensor of the infrared eye-tracker camera so as to detect said infrared-detected eye movements of the patient.
21. The computer-implemented method of claim 1 , wherein the cloud server comprises a web portal configured output an assessment report that displays one or more numeric index scores indicative of a severity of a developmental condition of the patient.