Visualization of images via an enhanced eye tracking system
In this patent, an improved eye tracking system is implemented for enhanced viewing. This system incorporates eye facing cameras and tracks display settings longitudinally and determines which portions of the image have been viewing and to what extent. A longitudinal dataset is generated and analyzed to better understand the human review process and cause improvements thereof. Furthermore, image review is enhanced by highlighting portions of the image that have not been adequately reviewed. Comparison across multiple reviewers is also performed to improve a user's performance.
1. A method comprising:
for a first time epoch, performing the steps comprising:
displaying an image onto a display with a first set of display settings;
using an eye facing camera to track a user's first fixation location on the display;
determining a first imaging finding located at the user's first fixation location on the display;
recording data in a dataset including the first imaging finding with a first set of display settings;
for a subsequent time epoch, performing the steps comprising:
displaying the image with a subsequent set of display settings
wherein the subsequent set of display settings is different from the first set of display settings;
using the eye facing camera to track the user's subsequent fixation location on the display;
determining a subsequent imaging finding located at the user's subsequent fixation location on the display
wherein the subsequent imaging finding comprises a segmented item,
wherein the segmented item is assigned an optimal viewing setting, and
wherein the subsequent fixation location at the subsequent imaging finding causes the subsequent imaging finding to be displayed with said optimal viewing setting; and
recording data in the dataset including the subsequent imaging finding with the subsequent set of display settings.
2. The method of claim 1 further comprising analyzing the dataset to determine the extent of the review by the user.
3. The method of claim 1 further comprising analyzing the dataset comprising generating metrics correlating fixation locations with imaging findings comprises of at least one of the group consisting of:
a number of fixation location(s) for an imaging finding;
an imaging plane of fixation location(s) for an imaging finding; and
a sequence of fixation location(s) for an imaging finding.
4. The method of claim 3 further comprising determining an average for each metric for the user and providing an notification to the user when a current exam's metric differs from the average.
5. The method of claim 3 further comprising determining an average for each metric for a population of users to develop a set of normative metrics and providing a notification to the user when the user's metric differs from the population.
6. The method of claim 1 further comprising comparing the dataset with a predetermined criteria to determine whether a minimum review has been completed wherein the predetermined criteria comprises of at least one of the group consisting of:
a minimum number of fixation locations for the image;
a minimum number of fixation locations for each imaging finding;
a minimum number of fixation locations for each subsegmented area within an imaging finding;
a minimum viewing time for each imaging feature;
a minimum number of imaging planes an imaging finding has fixation locations in situations comprising wherein the imaging dataset comprises cross-sectional imaging planes;
a determination of whether or not a structure had optimized display during a fixation location; and
a determination of whether or not a predetermined sequence of fixation locations for each imaging feature has been achieved.
7. The method of claim 6 further comprising altering the displayed image based on the relationship between the analyzed dataset and the predetermined criteria further comprising:
assigning a first visual representation adjustment logic to imaging finding(s) that have met the predetermined criteria; and
assigning a second visual representation adjustment logic to imaging finding(s) that have not met the predetermined criteria wherein the first visual representation adjustment logic is different from the second visual representation adjustment logic.
8. The method of claim 7 further comprising providing visual feedback to the user based on whether or not the predetermined criteria has been met to guide the user in the performance of a comprehensive review of the image.
9. The method of claim 1 further comprising wherein the display has a curvature comprising:
the top portion of the monitor curves inwards towards the user;
the bottom portion of the monitor curves inwards towards the user;
the left portion of the monitor curves inward towards the user; and
the right portion of the monitor curves inward towards the user.
10. The method of claim 1 further comprising utilizing a head display unit (HDU) comprising of at least one of the group consisting of: an extended reality display; eye display with shutters; and, eye display with polarized lenses.
11. The method of claim 1 further comprising using a keyboard comprising wherein the center of the keyboard is in an elevated position with respect to the sides of the keyboard.
12. The method of claim 1 further comprising wherein the display setting comprises at least one of the group consisting of: a pan setting; a zoom setting; and, a window/level setting.
13. The method of claim 1 further wherein the dataset comprises data on areas of the image included in the user's high resolution field of view.
14. The method of claim 1 further comprising using eye tracking data to implement at least one of the group comprising: a smart panning viewing option; a smart zoom viewing option; and a smart window/level viewing option.
15. The method of claim 1 further comprising analyzing the dataset by an artificial intelligence algorithm.
16. A non-transitory computer readable medium having computer readable code thereon for image processing, the medium comprising:
instructions for a first time epoch, performing the steps comprising:
displaying an image onto a display with a first set of display settings;
using an eye facing camera to track a user's first fixation location on the display;
determining a first imaging finding located at the user's first fixation location on the display;
recording data in a dataset including the first imaging finding with a first set of display settings;
instructions for a subsequent time epoch, performing the steps comprising:
displaying the image with a subsequent set of display settings wherein the subsequent set of display settings is different from the first set of display settings;
using the eye facing camera to track the user's subsequent fixation location on the display;
determining a subsequent imaging finding located at the user's subsequent fixation location on the display; and
recording data in the dataset including the subsequent imaging finding with the subsequent set of display settings;
instructions that compare the dataset with a predetermined criteria to determine whether a minimum review has been completed wherein the predetermined criteria comprises of at least one of the group consisting of:
a minimum number of fixation locations for the image;
a minimum number of fixation locations for each imaging finding;
a minimum number of fixation locations for each subsegmented area within an imaging finding;
a minimum viewing time for each imaging finding;
a minimum number of imaging planes an imaging finding has fixation locations in situations comprising wherein the imaging dataset comprises cross-sectional imaging planes;
a determination of whether or not a structure had optimized display during a fixation location; and
a determination of whether or not a predetermined sequence of fixation locations for each imaging feature has been achieved,
instructions that alter the displayed image based on the relationship between the analyzed dataset and the predetermined criteria further comprising:
assigning a first visual representation adjustment logic to imaging finding(s) that have met the predetermined criteria; and
assigning a second visual representation adjustment logic to imaging finding(s) that have not met the predetermined criteria where the first visual representation adjustment logic is different from the second visual representation adjustment logic; and
instructions that provide visual feedback to the user based on whether or not the predetermined criteria has been met to guide the user in the performance of a comprehensive review of the image.
17. A computer system comprising:
a memory;
a processor; and wherein the memory is encoded with an application providing image processing that when performed on the processor provides a process for processing information, the process causing the computer system to perform the operations of:
for a first time epoch, performing the steps comprising:
displaying an image onto a display with a first set of display settings;
using an eye facing camera to track a user's first fixation location on the display;
determining a first imaging finding located at the user's first fixation location on the display;
recording data in a dataset including the first imaging finding with a first set of display settings;
for a subsequent time epoch, performing the steps comprising:
displaying the image with a subsequent set of display settings wherein the subsequent set of display settings is different from the first set of display settings;
using the eye facing camera to track the user's subsequent fixation location on the display;
determining a subsequent imaging finding located at the user's subsequent fixation location on the display; and
recording data in the dataset including the subsequent imaging finding with the subsequent set of display settings
comparing the dataset with a predetermined criteria to determine whether a minimum review has been completed wherein the predetermined criteria comprises of at least one of the group consisting of,
a minimum number of fixation locations for the image;
a minimum number of fixation locations for each imaging finding;
a minimum number of fixation locations for each subsegmented area within an imaging finding;
a minimum viewing time for each imaging finding;
a minimum number of imaging planes an imaging finding has fixation locations in situations comprising wherein the imaging dataset comprises cross-sectional imaging planes;
a determination of whether or not a structure had optimized display during a fixation location; and
a determination of whether or not a predetermined sequence of fixation locations for each imaging feature has been achieved; and
instructions that alter the displayed image based on the relationship between the analyzed dataset and the predetermined criteria further comprising:
assigning a first visual representation adjustment logic to imaging finding(s) that have met the predetermined criteria, and
assigning a second visual representation adjustment logic to imaging finding(s) that have not met the predetermined criteria wherein the first visual representation adjustment logic is different from the second visual representation adjustment logic.