SYSTEMS AND METHODS FOR VIRTUAL REALITY, AUGMENTED REALITY, AND MIXED REALITY BASED VISUAL FUNCTION ASSESSMENT
Presented herein are systems and methods for improved virtual reality (VR), augmented reality (AR), and/or mixed reality (MR)-based visual function assessment. In various embodiments, systems and methods described herein utilize eye tracking to display static and/or changing/moving visual stimuli (targets) to the subject to elicit and record patterns of eye movements relative to the stimuli, where said patterns of eye movements are correlated to disease or retinal dysfunction. In certain embodiments, techniques of functional vision testing automatically perform, in real time, during the course of the functional vision test, a visibility determination to automatically identify if the subject is tracking the target in time and space. In some embodiments, systems and methods align VR headset optics with the wearer's eye and/or render a graphical scotoma mask that simulates the effect of a particular patient's scotoma on a visual field of either the patient or another individual.
1 . A method for conducting a functional vision test (e.g., a contrast sensitivity and/or spatial frequency test, e.g., a radial sweep test) on a subject using a virtual- and/or augmented- and/or mixed-reality device (e.g., VR headset with eye tracking capability), the method comprising:
rendering and displaying (e.g., on a head-mounted display of the VR headset) to the subject, by a processor of a computing device, over a course of the functional vision test, one or more targets (e.g., two or more, three or more, four or more, or five or more targets) within a virtual and/or augmented scene in a visual field of the subject;
automatically performing, by the processor, in real time, during the course of the functional vision test, a visibility determination to automatically identify if the subject is tracking the one or more targets (e.g., in time and space) within the virtual and/or augmented scene, wherein the visibility determination is based, at least in part, on time-based parameters; and
optionally, computing and/or displaying, by the processor, test results using the visibility determination.
2 . The method of claim 1 , wherein performing the visibility determination comprises:
determining, by the processor, in real time, a point of gaze of the subject; and
comparing, by the processor, the point of gaze to a current spatial position of the one or more targets within the virtual and/or augmented scene using the time-based parameters.
3 . The method of claim 1 or claim 2 , wherein performing the visibility determination comprises:
determining, by the processor, in real time, a point of gaze of the subject; and
determining, by the processor, whether the point of gaze is aligned with one of the target(s) within the virtual and/or augmented scene.
4 . The method of any one of claims 1-3 , wherein performing the visibility determination comprises:
determining, by the processor, in real time, a point of gaze of the subject; and
determining, by the processor, whether the subject is tracking one of the target(s) within the virtual and/or augmented scene using at least one of the time-based parameters.
5 . The method of any one of claims 2-4 , wherein the time-based parameters each correspond to a time period during which the point of gaze is or is not aligned with one or more of the one or more targets within the virtual and/or augmented scene.
6 . The method of any one of claims 2-5 , comprising automatically adjusting (e.g., decreasing) contrast and/or spatial resolution of one or more of the target(s) within the virtual and/or augmented scene based on the comparison of the point of gaze to the current spatial position of the one or more of the target(s) within the virtual and/or augmented scene (e.g., based on a determination, by the processor, that the point of gaze is aligned to the one or more of the target(s) for a (or at least a) predetermined threshold period of time using one or more of the time-based parameters).
7 . The method of any one of claims 1-6 , wherein there are a plurality of targets displayed within the virtual and/or augmented scene in the visual field of the subject, and wherein the time-based parameters comprises (e.g., consists of) four parameters: (i) for tracking when a (e.g., the) point of gaze of the subject is on (e.g., aligned with) a particular target of the plurality of targets within the virtual and/or augmented scene, (ii) for tracking when a (e.g., the) point of gaze of the subject is off (e.g., not aligned with) a particular target of the plurality of targets within the virtual and/or augmented scene, (iii) for tracking when a (e.g., the) point of gaze of the subject is on (e.g., aligned with) any of the plurality of targets within the virtual and/or augmented scene, and (iv) for tracking when a (e.g., the) point of gaze of the subject is off (e.g., not aligned with) all of the plurality of targets within the virtual and/or augmented scene.
8 . The method of claim 7 , wherein the four parameters account for at least 80% (e.g., at least 90%, e.g., at least 95%, e.g., at least 98%, e.g., 100%) of all meaningful variables used in the functional vision test (e.g., where a meaningful variable is a variable that has at least a 5% impact on the outcome of the functional vision test).
9 . The method of any one of claims 1-8 , wherein the time-based parameters are asymmetric [e.g., wherein parameter (i) and parameter (ii) refer to (e.g., correspond to) different time lengths and/or wherein parameter (iii) and parameter (iv) refer to (e.g., correspond to) different time lengths (e.g., wherein parameter (i) refers to a shorter time length than parameter (ii) does and/or parameter (iii) refers to a shorter time length than parameter (iv) does].
10 . The method of any one of claims 1-9 , wherein (i) at least one of the time-based parameters corresponds to the subject tracking one of the target(s) [e.g., during which a point of gaze is aligned with (e.g., incident on) the one of the target(s) within the virtual and/or augmented scene], (ii) at least one of the time-based parameters corresponds to the subject tracking any of the target(s) [e.g., during which a point of gaze is aligned with (e.g., incident on) any of the target(s) within the virtual and/or augmented scene], or (iii) both (i) and (ii).
11 . The method of any one of claims 1-10 , comprising automatically adjusting contrast, spatial position, spatial resolution, and/or direction of movement of the one or more targets (e.g., abruptly) (e.g., one or more of the one or more targets) within the virtual and/or augmented scene during the test using one or more of the time-based parameters (e.g., making a tracked one of the target(s) lower contrast to increase difficulty, changing direction of movement of a tracked one of the target(s) in an abrupt way to increase difficulty, resetting all of the target(s) with higher contrast, or stopping the test).
12 . The method of claim 11 , wherein adjusting the contrast, spatial position, spatial resolution, and/or direction of movement of the one or more targets comprises:
determining, by the processor, in real time, a point of gaze of the subject; and
automatically adjusting the contrast and/or spatial resolution of the one or more targets (e.g., of at least one of the one or more targets) within the virtual and/or augmented scene based at least in part on a point of gaze of the subject being aligned with the one or more targets according to at least one of the time-based parameters.
13 . The method of any one of claims 1-12 , wherein the visibility determination is performed using only the time-based parameters (i.e., no non-time based parameters are used to make the visibility determination).
14 . The method of any one of claims 1-13 , wherein no more than 10 total parameters (e.g., no more than 8 total parameters, no more than 6 total parameters, no more than 5 total parameters, or no more than 4 total parameters) are used to perform the visibility determination (e.g., and each of the parameters is a time-based parameter).
15 . The method of any one of claims 1-14 , wherein the one or more targets move and/or change direction of movement within the virtual and/or augmented scene during the functional vision test [e.g., regardless of whether the subject is or is not tracking the target(s) (e.g., as determined, by the processor, using a point of gaze of the subject)].
16 . The method of any one of claims 1-15 , wherein the one or more targets change contrast within the virtual and/or augmented scene during the functional vision test (e.g., based on determining, by the processor, in real time, that the subject is tracking the one or more targets) [e.g., changing contrast and/or spatial resolution of a target only when the subject has been tracking (e.g., continuously or intermittently) the target for a predetermined period of time)].
17 . The method of any one of claims 1-16 , wherein the functional vision test is a test (e.g., an outcome measure, e.g., a functional endpoint) for one or more eye conditions selected from the group consisting of: diabetic retinopathy (e.g., with or without diabetic macular retinopathy), Stargardt disease, Leber hereditary optic neuropathy (LHON), retinitis pigmentosa, glaucoma, inner nuclear layer disease, geographic atrophy, and age-related macular degeneration (AMD) (e.g., intermediate age-related macular degeneration (intermediate AMD), dry age-related macular degeneration (dry AMD), wet age-related macular degeneration (wet AMD), and/or neovascular age-related macular degeneration (neovascular AMD)).
18 . The method of any one of claims 1-17 , wherein the tests results are an outcome measure for an eye condition (e.g., affecting one or both eyes of the subject).
19 . The method of claim 18 , wherein the eye condition is selected from the group consisting of: diabetic retinopathy (e.g., with or without diabetic macular retinopathy), Stargardt disease, Leber hereditary optic neuropathy (LHON), retinitis pigmentosa, glaucoma, inner nuclear layer disease, geographic atrophy, and age-related macular degeneration (AMD) (e.g., intermediate age-related macular degeneration (intermediate AMD), dry age-related macular degeneration (dry AMD), wet age-related macular degeneration (wet AMD), and/or neovascular age-related macular degeneration (neovascular AMD)).
20 . The method of any one of claims 1-19 , wherein the test results indicate presence of, severity of, and/or progression of an eye condition of the subject [e.g., diabetic retinopathy (e.g., with or without diabetic macular retinopathy), Stargardt disease, Leber hereditary optic neuropathy (LHON), retinitis pigmentosa, glaucoma, inner nuclear layer disease, geographic atrophy, and age-related macular degeneration (AMD) (e.g., intermediate age-related macular degeneration (intermediate AMD), dry age-related macular degeneration (dry AMD), wet age-related macular degeneration (wet AMD), and/or neovascular age-related macular degeneration (neovascular AMD))].
21 . The method of any one of claims 1-20 , wherein the test results comprise a metric that corresponds to functional vision of the subject (e.g., an area under curve (AUC) metric) (e.g., having at least a 90% sensitivity at 100% specificity, at least 91% sensitivity at 100% specificity, at least 92% sensitivity at 100% specificity, or at least 92.5% sensitivity at 100% specificity) (e.g., having at least a 90% sensitivity, at least 91% sensitivity, at least 92% sensitivity, or at least 92.5% sensitivity).
22 . The method of claim 21 , wherein the metric is a sparse AUC metric (e.g., wherein one or more radial sweeps have not been performed and/or not considered).
23 . The method of any one of claims 1-22 , wherein each of the one or more targets is a visibility patch (e.g., a contrast-based visibility patch) graphically rendered within the virtual and/or augmented scene.
24 . The method of any one of claims 1-23 , wherein the method is performed without use of artificial intelligence.
25 . The method of any one of claims 1-24 , wherein the test results comprise a function of contrast sensitivity (e.g., inverse of root-mean-square (RMS) contrast ratio) and spatial frequency (in cycles per degree, CPD) (e.g., stored as or presented in a plot).
26 . The method of any one of claims 1-25 , comprising simulating, by the processor, a scotoma during the functional vision test (e.g., using the method of any one of claims 84 - 87 ).
27 . A packaged pharmaceutical composition or kit comprising a pharmaceutically acceptable vessel, a therapeutic agent secured or otherwise sealed within the vessel, and a label, wherein the therapeutic agent is for an eye condition [e.g., geographic atrophy and/or an age-related macular degeneration (AMD) (e.g., intermediate age-related macular degeneration (intermediate AMD), dry age-related macular degeneration (dry AMD), wet age-related macular degeneration (wet AMD), and/or neovascular age-related macular degeneration (neovascular AMD))], and the label comprises a description and/or code that identifies that the therapeutic agent is for treatment of the eye condition, (i) the eye condition having been diagnosed and/or monitored by a functional vision test performed according to the method of any one of claims 1-26 and/or (ii) efficacy of the therapeutic agent having been established and/or confirmed in a population of subjects (e.g., patients) by a functional vision test performed according to the method of any one of claims 1-26 .
28 . A method of treating a subject that has been diagnosed with an eye condition, has been or is monitored for an eye conditions, and/or has been determined to exhibit a worsening severity of an eye condition (e.g., affecting one or both eyes of the subject) using a functional vision test according to the method of any one of claims 1-26 , the method comprising administering a therapeutically effective amount of a therapeutic agent (e.g., pharmaceutical compound) to the subject.
29 . The method of claim 28 , wherein the eye condition is selected from the group consisting of: diabetic retinopathy (e.g., with or without diabetic macular retinopathy), Stargardt disease, Leber hereditary optic neuropathy (LHON), retinitis pigmentosa, glaucoma, inner nuclear layer disease, geographic atrophy, and age-related macular degeneration (AMD) (e.g., intermediate age-related macular degeneration (intermediate AMD), dry age-related macular degeneration (dry AMD), wet age-related macular degeneration (wet AMD), and/or neovascular age-related macular degeneration (neovascular AMD)).
30 . The method of claim 28 or claim 29 , wherein the therapeutic agent is an antibody (e.g., a monoclonal antibody) (e.g., an anti-factor D antibody or a vascular endothelial growth factor (VEGF) inhibitor).
31 . The method of claim 28 or claim 29 , wherein the therapeutic agent is a vascular endothelial growth factor (VEGF) inhibitor.
32 . The method of any one of claims 28-31 , wherein the therapeutic agent is ranibizumab, faricimab, brolucizumab, aflibercept, or pegaptanib.
33 . The method of claim 28 or claim 29 , wherein the therapeutic agent comprises a vitamin supplement and/or mineral supplement (e.g., comprising vitamin C, zinc, vitamin E, copper, or beta-carotene).
34 . The method of claim 28 or claim 29 , wherein the therapeutic agent comprises a complement inhibitor (e.g., a C3 inhibitor or a C5 inhibitor).
35 . The method of claim 34 , wherein the complement inhibitor comprises a peptide, protein, antibody, or aptamer that binds to C3 and/or a biologically active fragment of C3 (e.g., C3b or C3a).
36 . The method of claim 28 or claim 29 , wherein the therapeutic agent is (i) a vitamin supplement and/or mineral supplement selected from the group consisting of vitamin C, zinc, vitamin E, copper, beta-carotene, and combinations thereof, (ii) ranibizumab, (iii) faricimab, (iv) brolucizumab, (v) aflibercept, (vi) pegaptanib, (vii) a complement inhibitor, (viii) a neuroprotective agent, (ix) an anti-inflammatory agent, (x) a free radical scavenger, (xi) an anti-apoptotic agent, (xii) an integrin modulator, (xiii) a gene therapy, or (xiv) a cell therapy.
37 . A method of determining therapeutic effectiveness (e.g., benefit) of a therapeutic agent, the method comprising (i) administering a functional vision test according to the method of any one of claims 1-26 to a subject and (ii) determining therapeutic effectiveness (e.g., benefit) of a therapeutic agent to the subject based on test results from the functional vision test.
38 . The method of claim 37 , wherein determining the therapeutic effectiveness comprises quantifying quality adjusted life years (QALY).
39 . The method of claim 37 or claim 38 , comprising determining cost effectiveness of the therapeutic agent based, at least in part, on determining the therapeutic effectiveness based on test results from the functional vision test (e.g., based on quantifying QALY).
40 . The method of any one of claims 37-39 , wherein the therapeutic agent is (i) a vitamin supplement and/or mineral supplement selected from the group consisting of vitamin C, zinc, vitamin E, copper, beta-carotene, and combinations thereof, (ii) ranibizumab, (iii) faricimab, (iv) brolucizumab, (v) aflibercept, (vi) pegaptanib, (vii) a complement inhibitor, (viii) a neuroprotective agent, (ix) an anti-inflammatory agent, (x) a free radical scavenger, (xi) an anti-apoptotic agent, (xii) an integrin modulator, (xiii) a gene therapy, or (xiv) a cell therapy.
41 . Use of (i) a vitamin supplement and/or mineral supplement selected from the group consisting of vitamin C, zinc, vitamin E, copper, beta-carotene, and combinations thereof, (ii) ranibizumab, (iii) faricimab, (iv) brolucizumab, (v) aflibercept, (vi) pegaptanib, (vii) a complement inhibitor, (viii) a neuroprotective agent, (ix) an anti-inflammatory agent, (x) a free radical scavenger, (xi) an anti-apoptotic agent, (xii) an integrin modulator, (xiii) a gene therapy, or (xiv) a cell therapy, for treatment of a subject diagnosed with and/or monitored for an eye condition using the method according to any one of claims 1-26 .
42 . A method of treating a subject that has been diagnosed with an eye condition, has been or is monitored for an eye conditions, and/or has been determined to exhibit a worsening severity of an eye condition (e.g., affecting one or both eyes of the subject) using a functional vision test according to the method of any one of claims 1-26 , the method comprising administering a therapeutically effective therapeutic intervention (e.g., laser coagulation therapy) to the subject.
43 . A method of treating a subject that has been diagnosed with an eye condition, has been or is monitored for an eye conditions, and/or has been determined to exhibit a worsening severity of an eye condition (e.g., affecting one or both eyes of the subject), the method comprising administering to the subject a therapeutically effective amount of a therapeutic agent (e.g., pharmaceutical compound), wherein the efficacy of the therapeutic agent has been established or confirmed in a population of subjects using a functional vision test according to the method of any one of claims 1-26 .
44 . The method of claim 43 , wherein the eye condition is selected from the group consisting of: diabetic retinopathy (e.g., with or without diabetic macular retinopathy), Stargardt disease, Leber hereditary optic neuropathy (LHON), retinitis pigmentosa, glaucoma, inner nuclear layer disease, geographic atrophy, and age-related macular degeneration (AMD) (e.g., intermediate age-related macular degeneration (intermediate AMD), dry age-related macular degeneration (dry AMD), wet age-related macular degeneration (wet AMD), and/or neovascular age-related macular degeneration (neovascular AMD)).
45 . The method of claim 43 or claim 44 , wherein treatment with the therapeutic agent preserves or improves performance on the functional vision test and/or reduces rate of deterioration in performance on the functional vision test as compared to a suitable control [e.g., no treatment or sham treatment (e.g., placebo)].
46 . Use of a therapeutic agent for treatment of an individual diagnosed with an eye condition [e.g., geographic atrophy and/or an age-related macular degeneration (AMD) (e.g., intermediate age-related macular degeneration (intermediate AMD), dry age-related macular degeneration (dry AMD), wet age-related macular degeneration (wet AMD), and/or neovascular age-related macular degeneration (AMD)] wherein therapeutic efficacy of the therapeutic agent has been established or confirmed using a method according to any one of claims 1-26 in a population of subjects.
47 . The use of claim 46 , wherein the therapeutic agent comprises (i) a vitamin supplement and/or mineral supplement selected from the group consisting of vitamin C, zinc, vitamin E, copper, beta-carotene, and combinations thereof, (ii) ranibizumab, (iii) faricimab, (iv) brolucizumab, (v) aflibercept, (vi) pegaptanib, (vii) a complement inhibitor, (viii) a neuroprotective agent, (ix) an anti-inflammatory agent, (x) a free radical scavenger, (xi) an anti-apoptotic agent, (xii) an integrin modulator, (xiii) a gene therapy, or (xiv) a cell therapy.
48 . A system [e.g., a virtual- and/or augmented- and/or mixed-reality device (e.g., VR headset with eye tracking capability)], comprising a processor and a memory having instructions stored thereon, the instructions executable by the processor to perform the method of any one of claims 1-26 .
49 . A method for conducting a vision test on a subject using a virtual- and/or augmented- and/or mixed-reality device (e.g., a VR headset with eye tracking capability), the method comprising rendering and displaying to the subject, by a processor, an object within a virtual and/or augmented scene in a visual field of the subject via the device (e.g., on a head-mounted display of the VR headset), wherein a position of the object remains fixed in a virtual space (e.g., fixed within the virtual and/or augmented scene) when a head of the subject changes orientation [e.g., such that the subject can orient the object in a preferred location relative to a point of gaze of the subject (e.g., a location corresponding to alignment of best vision with an area of interest in the object)].
50 . The method of claim 49 , wherein the subject translating the device does not move the object relative to the subject in the virtual space (e.g., the subject cannot get closer or further from the object in the virtual space).
51 . The method of claim 49 or claim 50 , wherein the object (e.g., as rendered on a head-mounted display of the VR headset) is curved (i.e., not flat) (e.g., a curved test chart) (e.g., to account for diminished peripheral vision due to quality of one or more lenses of the device).
52 . The method of any one of claims 49-51 , wherein the object is a virtual test chart (e.g., eye chart) (e.g., comprising one or more targets, e.g., moving target(s), e.g., for a functional vision test).
53 . The method of any one of claims 49-52 , wherein the VR headset is a low-cost, off-the-shelf headset.
54 . A system [e.g., a virtual- and/or augmented- and/or mixed-reality device (e.g., VR headset with eye tracking capability) (e.g., a low-cost, off-the-shelf VR headset)], comprising a processor and a memory having instructions stored thereon, the instructions executable by the processor to perform the method of any one of claims 49-52 .
55 . A system for identifying a preferred retinal locus (PRL) relative to retina anatomy (e.g., a locus in an anatomy reference system) for one or both eyes of a subject (e.g., a patient with a macular disease such as macular degeneration, e.g., a patient with a central scotoma) (e.g., wherein the preferred retinal locus is a position on the retina other than the fovea or macula), the system comprising:
a processor of a computing device; and
a memory having instructions stored thereon, wherein the instructions, when executed by the processor, cause the processor to:
receive a first data stream (e.g., from a VR headset) corresponding to a gaze direction of an eye of the subject (e.g., said gaze direction defining a visual axis) in a headset reference system (e.g., independent of actual eye anatomy) over time;
receive a second data stream (e.g., from the VR headset) corresponding to a gaze origin at a nodal point of the eye of the subject (e.g., said nodal point corresponding to a center of corneal curvature of the eye) in a headset reference system over time, wherein the nodal point moves around a center of eye rotation as the eye rotates to change gaze direction;
identify a geometric volume (e.g., a sphere, e.g., a best fit sphere) having a reference point (e.g., a center) corresponding to the center of eye rotation and having a surface approximated by the nodal points, using data from the second data stream;
identify an anatomical reference (e.g., an optical axis of the eye) from the identified geometric volume (e.g., the sphere, e.g., the best fit sphere) (e.g., identify the optical axis of the eye as a line connecting center of the sphere and gaze origin); and
identify the preferred retinal locus (PRL) relative to retina anatomy (e.g., a locus in the anatomy reference system) using the identified anatomical reference (e.g., the optical axis of the eye) and data from the first data stream (e.g., use data from the first data stream to calculate a horizontal angle and/or vertical angle—φ, θ—between the optical axis and visual axis of the eye, and identify the PRL using the identified optical axis of the eye and the calculated horizontal angle φ and/or vertical angle θ) [e.g., and monitor the PRL (e.g., the angles φ, θ), in multiple sessions with the subject performed over time (e.g., months or years) to detect PRL changes, e.g., as disease progresses].
56 . The system of claim 55 , comprising a virtual- and/or augmented- and/or mixed-reality headset for producing the first data stream and the second data stream.
57 . The system of claim 56 , comprising an eye-tracking camera (e.g., wherein the headset comprises the eye-tracking camera).
58 . The system of claim 57 , comprising an illumination source (e.g., an infrared illumination source) for illuminating (the) one or both eyes of the subject (e.g., wherein the headset comprises the illumination source).
59 . The system of any one of claims 56 to 58 , wherein the virtual- and/or augmented- and/or mixed-reality headset comprises one or more members selected from the group consisting of: a head-mounted display, one or more lenses, one or more headset processors for producing the first data stream and/or the second data stream [e.g., wherein the processor of the computing device that executes the instructions in claim 1 is any one or more of (i) to (iv) as follows: (i) a portion or all of the one or more headset processors, (ii) distinct from the one or more headset processors, (iii) at least partially co-located with the one or more headset processors, (iv) spatially separated (remote) from the one or more headset processors, and (v) in electrical and/or data communication with the one or more headset processors], one or more mechanisms (e.g., dial, toggle, knob, or switch) for physically adjusting the position of the display in the headset, one or more mechanisms (e.g., mechanical mechanisms, e.g., knobs, straps, dials, or the like) for adjusting (e.g., manually adjusting) the horizontal and/or vertical position of the headset relative to the head of the subject, a circuit board, and a head support (e.g., strap(s), mount(s), brace(s), and/or other physical structure(s) to stabilize the headset on the head of the subject).
60 . The system of any one of claims 55 to 59 , wherein the instructions, when executed by the processor, cause the processor to display visual stimuli (e.g., one or more static and/or changing/moving graphical targets) to the subject to elicit and record patterns of eye movements relative to the stimuli using the identified PRL, where said recorded patterns of eye movements are correlated to a disease or condition (e.g., a retinal dysfunction).
61 . The system of claim 60 , wherein the instructions, when executed by the processor, cause the processor to identify said subject may have said disease or condition based at least in part on said recorded patterns of eye movements (e.g., and wherein the instructions, when executed by the processor, cause the processor to present a graphical display (e.g., an alphanumeric) indicating the subject may have said disease or condition).
62 . A system for prompting adjustment (e.g., physical adjustment, e.g., manual adjustment by the wearer) of a virtual- and/or augmented- and/or mixed-reality headset position relative to a wearer's head (e.g., to improve/optimize alignment of the center of the headset lens(es) with the center of the (respective) eye(s) of the wearer), the system comprising:
a virtual- and/or augmented- and/or mixed-reality headset with one or more mechanisms (e.g., mechanical mechanisms, e.g., knobs, straps, dials, or the like) for adjusting (e.g., manually adjusting) the vertical position of the headset relative to the head of the wearer;
a processor of a computing device; and
a memory having instructions stored thereon, wherein the instructions, when executed by the processor, cause the processor to:
receive a first data stream corresponding to wearer eye position relative to the headset (e.g., the first data stream corresponding to gaze origin at a nodal point of (each of) one or both eyes of the wearer, each said nodal point corresponding to a center of corneal curvature of the (respective) eye) (e.g., the position of the center of one or both eyes of the wearer relative to the center of one or both respective headset lenses) over time, wherein a position of a virtual camera is linked to the wearer eye position relative to the headset; and
render and display a virtual iron sight in a visual field of the headset wearer in real time (or near real time) corresponding to real-time (or near real time) position of the virtual camera as the first data stream is received, said iron sight comprising two concentric rings having fixed position relative to each other and a third ring having a color and/or tint that contrasts with the two concentric rings, said third ring having a visually detectable offset relative to the two concentric rings when a center of one or both respective headset lens(es) is/are misaligned with the gaze origin(s) (center(s) of corneal curvature) of the respective eye(s) of the wearer,
wherein an offset of the third ring relative to the two concentric rings as it appears to the wearer in the visual field of the headset prompts physical adjustment (e.g., by the wearer) of the vertical position of the headset via the one or more (e.g., mechanical) mechanisms until said adjustment causes the center of one or both respective headset lens(es) to be aligned with the gaze origin(s) of the respective eye(s) of the wearer, and the resulting position of the virtual camera causes the third ring to be displayed entirely between the two concentric rings.
63 . The system of claim 62 , wherein the virtual- and/or augmented- and/or mixed-reality headset produces the first data stream.
64 . The system of claim 63 , comprising an eye-tracking camera (e.g., wherein the headset comprises the eye-tracking camera).
65 . The system of claim 64 , comprising an illumination source (e.g., an infrared illumination source) for illuminating (the) one or both eyes of the wearer (e.g., wherein the headset comprises the illumination source).
66 . The system of any one of claims 63 to 65 , wherein the virtual- and/or augmented- and/or mixed-reality headset comprises one or more members selected from the group consisting of: a head-mounted display, one or more lenses, one or more mechanisms (e.g., dial, toggle, knob, or switch) for physically adjusting the position of the display in the headset, a circuit board, and a head support (e.g., strap(s), mount(s), brace(s), and/or other physical structure(s) to stabilize the headset on the head of the subject).
67 . The system of any one of claims 62 to 66 , wherein the instructions, when executed by the processor, cause the processor to—following said adjustment that causes the center of one or both respective headset lens(es) to be aligned with the gaze origin(s) of the respective eye(s) of the wearer—display visual stimuli (e.g., one or more static and/or changing/moving graphical targets) to the subject to elicit and record patterns of eye movements relative to the stimuli, where said recorded patterns of eye movements are correlated to a disease or condition (e.g., a retinal dysfunction).
68 . The system of claim 67 , wherein the instructions, when executed by the processor, cause the processor to identify said subject may have said disease or condition based at least in part on said recorded patterns of eye movements (e.g., and wherein the instructions, when executed by the processor, cause the processor to present a graphical display (e.g., an alphanumeric) indicating the subject may have said disease or condition).
69 . A system for rendering a graphical (e.g., 2D) scotoma mask that simulates the effect of a scotoma on a visual field, said scotoma suffered by a particular patient, the system comprising:
a processor of a computing device; and
a memory having instructions stored thereon, wherein the instructions, when executed by the processor, cause the processor to:
receive data corresponding to a shape of a simulated scotoma (e.g., a selected default scotoma shape such as a disc, or a personalized scotoma shape derived from microperimetry measurement of a subject, e.g., a differential light sensitivity (DLS) map);
receive input (e.g., from the particular patient suffering from a real scotoma) to identify one or more visual effects parameters corresponding to values of visual effects caused by the scotoma (e.g., said one or more parameters comprising one or more members selected from the group consisting of opacity, color saturation, blur, and distortion (e.g., pincushion, barrel, or whirl));
define the graphical scotoma mask according to the shape of the simulated scotoma and the one or more visual effects parameters;
receive a first data stream (e.g., from a VR headset) corresponding to a point of gaze of the wearer of the VR headset; and
render and display in real time (or near real time) to the wearer of the VR headset a virtual and/or augmented scene in the visual field of the wearer, at least a portion of said virtual and/or augmented scene modified according to the graphical scotoma mask, said virtual and/or augmented scene following a point of gaze of the wearer as said point of gaze changes in real time and as said virtual and/or augmented scene is affected by the simulated scotoma (e.g., wherein the wearer may be the patient or wherein the wearer may be an individual different from the patient).
70 . The system of claim 69 , wherein the instructions cause the processor to identify the one or more visual effects parameters from patient input by:
for a first period of time, (i) blocking vision of a healthy eye of the patient and (ii) displaying a virtual scene to the scotoma-affected eye of the patient or allowing viewing of a real field of view by the scotoma-affected eye of the patient, said patient having a one-sided scotoma;
for a second period of time, rendering and displaying (e.g., via the VR headset) the virtual scene and/or an augmented scene to only a (single) healthy eye of the patient a simulated scotoma, said virtual and/or augmented scene modified according to a graphical scotoma mask corresponding to a given shape and one or more adjustable visual effects parameters; and
updating the virtual and/or augmented scene according to feedback from the patient and rendering and displaying the updated virtual and/or augmented scene to the healthy eye of the patient in real time (or near real time), such that the patient may compare the patient's field of view in each eye and adjust the one or more visual effects parameters (and/or the scotoma shape) to match the field of view as seen by the eye with the real scotoma with the field of view as seen by the eye with the simulated scotoma.
71 . The system of claim 69 or 70 , comprising a virtual- and/or augmented- and/or mixed-reality headset which produces the first data stream.
72 . The system of any one of claims 69 to 71 , comprising an eye-tracking camera (e.g., wherein the headset comprises the eye-tracking camera).
73 . The system of any one of claims 69 to 72 , comprising an illumination source (e.g., an infrared illumination source) for illuminating (the) one or both eyes of the patient (e.g., wherein the headset comprises the illumination source).
74 . The system of any one of claims 69 to 73 , wherein the virtual- and/or augmented- and/or mixed-reality headset comprises one or more members selected from the group consisting of: a head-mounted display, one or more lenses, one or more mechanisms (e.g., dial, toggle, knob, or switch) for physically adjusting the position of the display in the headset, one or more mechanisms (e.g., mechanical mechanisms, e.g., knobs, straps, dials, or the like) for adjusting (e.g., manually adjusting) the horizontal and/or vertical position of the headset relative to the head of the subject, a circuit board, and a head support (e.g., strap(s), mount(s), brace(s), and/or other physical structure(s) to stabilize the headset on the head of the subject).
75 . The system of any one of claims 69 to 74 , wherein the instructions, when executed by the processor, cause the processor to display visual stimuli (e.g., one or more static and/or changing/moving graphical targets) to the subject to elicit and record patterns of eye movements relative to the stimuli, where said recorded patterns of eye movements are correlated to a disease or condition (e.g., a retinal dysfunction).
76 . The system of claim 75 , wherein the instructions, when executed by the processor, cause the processor to identify said wearer may have (or may have a risk of) said disease or condition based at least in part on said recorded patterns of eye movements (e.g., and wherein the instructions, when executed by the processor, cause the processor to present a graphical display (e.g., an alphanumeric) indicating the wearer may have (or may have a risk of) said disease or condition).
77 . A method for identifying a preferred retinal locus (PRL) relative to retina anatomy (e.g., a locus in an anatomy reference system) for one or both eyes of a subject (e.g., a patient with a macular disease such as macular degeneration, e.g., a patient with a central scotoma) (e.g., wherein the preferred retinal locus is a position on the retina other than the fovea or macula), the method comprising:
receiving, by a processor of a computing device, a first data stream (e.g., from a VR headset) corresponding to a gaze direction of an eye of the subject (e.g., said gaze direction defining a visual axis) in a headset reference system (e.g., independent of actual eye anatomy) over time;
receiving, by the processor, a second data stream (e.g., from the VR headset) corresponding to a gaze origin at a nodal point of the eye of the subject (e.g., said nodal point corresponding to a center of corneal curvature of the eye) in a headset reference system over time, wherein the nodal point moves around a center of eye rotation as the eye rotates to change gaze direction;
identifying, by the processor, a geometric volume (e.g., a sphere, e.g., a best fit sphere) having a reference point (e.g., a center) corresponding to the center of eye rotation and having a surface approximated by the nodal points, using data from the second data stream;
identifying, by the processor, an anatomical reference (e.g., an optical axis of the eye) from the identified geometric volume (e.g., the sphere, e.g., the best fit sphere) (e.g., identify the optical axis of the eye as a line connecting center of the sphere and gaze origin); and
identifying the preferred retinal locus (PRL) relative to retina anatomy (e.g., a locus in the anatomy reference system) using the identified anatomical reference (e.g., the optical axis of the eye) and data from the first data stream (e.g., use data from the first data stream to calculate a horizontal angle and/or vertical angle—φ, θ—between the optical axis and visual axis of the eye, and identify the PRL using the identified optical axis of the eye and the calculated horizontal angle φ and/or vertical angle θ) [e.g., and monitor the PRL (e.g., the angles φ, θ), in multiple sessions with the subject performed over time (e.g., months or years) to detect PRL changes, e.g., as disease progresses].
78 . The method of claim 77 , comprising displaying visual stimuli (e.g., one or more static and/or changing/moving graphical targets) to the subject to elicit and record (e.g., by the processor) patterns of eye movements relative to the stimuli using the identified PRL, where said recorded patterns of eye movements are correlated to a disease or condition (e.g., a retinal dysfunction).
79 . The method of claim 78 , comprising identifying, by the processor, said subject as having (e.g., or, alternatively, identifying said subject as having a risk of) said disease or condition based at least in part on said recorded patterns of eye movements (e.g., and presenting, by the processor, a graphical display (e.g., an alphanumeric) indicating the subject has (e.g., or, alternatively, may have) said disease or condition).
80 . A method for prompting adjustment (e.g., physical adjustment, e.g., manual adjustment by the wearer) of a virtual- and/or augmented- and/or mixed-reality headset position relative to a wearer's head (e.g., to improve/optimize alignment of the center of the headset lens(es) with the center of the (respective) eye(s) of the wearer), the method comprising:
receiving, by a processor of a computing device, a first data stream corresponding to wearer eye position relative to the headset (e.g., the first data stream corresponding to gaze origin at a nodal point of (each of) one or both eyes of the wearer, each said nodal point corresponding to a center of corneal curvature of the (respective) eye) (e.g., the position of the center of one or both eyes of the wearer relative to the center of one or both respective headset lenses) over time, wherein a position of a virtual camera is linked to the wearer eye position relative to the headset, wherein the headset comprises one or more mechanisms (e.g., mechanical mechanisms, e.g., knobs, straps, dials, or the like) for adjusting (e.g., manually adjusting) the vertical position of the headset relative to the head of the wearer; and
rendering and displaying, by the processor, a virtual iron sight in a visual field of the headset wearer in real time (or near real time) corresponding to real-time (or near real time) position of the virtual camera as the first data stream is received, said iron sight comprising two concentric rings having fixed position relative to each other and a third ring having a color and/or tint that contrasts with the two concentric rings, said third ring having a visually detectable offset relative to the two concentric rings when a center of one or both respective headset lens(es) is/are misaligned with the gaze origin(s) (center(s) of corneal curvature) of the respective eye(s) of the wearer,
wherein an offset of the third ring relative to the two concentric rings as it appears to the wearer in the visual field of the headset prompts physical adjustment (e.g., by the wearer) of the vertical position of the headset via the one or more (e.g., mechanical) mechanisms until said adjustment causes the center of one or both respective headset lens(es) to be aligned with the gaze origin(s) of the respective eye(s) of the wearer, and the resulting position of the virtual camera causes the third ring to be displayed entirely between the two concentric rings.
81 . The method of claim 80 , comprising checking virtual iron sign alignment during a visual function test.
82 . The method of claim 80 or 81 , comprising, following said adjustment that causes the center of one or both respective headset lens(es) to be aligned with the gaze origin(s) of the respective eye(s) of the wearer, displaying (e.g., by the processor) visual stimuli (e.g., one or more static and/or changing/moving graphical targets) to the wearer to elicit and record patterns of eye movements relative to the stimuli, where said recorded patterns of eye movements are correlated to a disease or condition (e.g., a retinal dysfunction).
83 . The method of claim 82 , comprising identifying, by the processor, that said wearer has (or may have) said disease or condition based at least in part on said recorded patterns of eye movements (e.g., and presenting a graphical display (e.g., an alphanumeric) indicating the wearer has (or may have) said disease or condition).
84 . A method for rendering a graphical (e.g., 2D) scotoma mask that simulates the effect of a scotoma on a visual field, said scotoma suffered by a particular patient, the method comprising:
receiving, by the processor of a computing device, data corresponding to a shape of a simulated scotoma (e.g., a selected default scotoma shape such as a disc, or a personalized scotoma shape derived from microperimetry measurement of a subject, e.g., a differential light sensitivity (DLS) map);
receiving, by the processor, input (e.g., from the particular patient suffering from a real scotoma) to identify one or more visual effects parameters corresponding to values of visual effects caused by the scotoma (e.g., said one or more parameters comprising one or more members selected from the group consisting of opacity, color saturation, blur, and distortion (e.g., pincushion, barrel, or whirl));
defining, by the processor, the graphical scotoma mask according to the shape of the simulated scotoma and the one or more visual effects parameters;
receiving, by the processor, a first data stream (e.g., from a VR headset) corresponding to a point of gaze of the wearer of the VR headset; and
rendering and displaying, by the processor, in real time (or near real time) to the wearer of the VR headset a virtual and/or augmented scene in the visual field of the wearer, at least a portion of said virtual and/or augmented scene modified according to the graphical scotoma mask, said virtual and/or augmented scene following a point of gaze of the wearer as said point of gaze changes in real time and as said virtual and/or augmented scene is affected by the simulated scotoma (e.g., wherein the wearer may be an individual different from the patient).
85 . The method of claim 84 , comprising identifying, by the processor, the one or more visual effects parameters from patient input by:
for a first period of time, (i) blocking vision of a healthy eye of the patient and (ii) displaying a virtual scene to the scotoma-affected eye of the patient or allowing viewing of a real field of view by the scotoma-affected eye of the patient, said patient having a one-sided scotoma;
for a second period of time, rendering and displaying (e.g., via the VR headset) the virtual scene and/or an augmented scene to only a (single) healthy eye of the patient a simulated scotoma, said virtual and/or augmented scene modified according to a graphical scotoma mask corresponding to a given shape and one or more adjustable visual effects parameters; and
updating the virtual and/or augmented scene according to feedback from the patient and rendering and displaying the updated virtual and/or augmented scene to the healthy eye of the patient in real time (or near real time), such that the patient may compare the patient's field of view in each eye and adjust the one or more visual effects parameters (and/or the scotoma shape) to match the field of view as seen by the eye with the real scotoma with the field of view as seen by the eye with the simulated scotoma.
86 . The method of claim 84 or 85 , comprising displaying (e.g., by the processor) visual stimuli (e.g., one or more static and/or changing/moving graphical targets) to the wearer to elicit and record patterns of eye movements relative to the stimuli, where said recorded patterns of eye movements are correlated to a disease or condition (e.g., a retinal dysfunction).
87 . The method of claim 86 , comprising identifying, by the processor, said wearer may have (or may have a risk of) said disease or condition based at least in part on said recorded patterns of eye movements (e.g., and presenting a graphical display (e.g., an alphanumeric) indicating the wearer may have (or may have a risk of) said disease or condition).
88 . A system for conducting a functional vision test (e.g., a contrast sensitivity and/or spatial frequency test, e.g., a radial sweep test) on a subject using a virtual- and/or augmented- and/or mixed-reality device (e.g., VR headset with eye tracking capability), the system comprising:
a processor of a computing device; and
a memory having instructions stored thereon, wherein the instructions, when executed by the processor, cause the processor to:
render and display (e.g., via the VR headset) to the subject, over a course of the functional vision test, a target (e.g., a moving target, e.g., a target that changes in spatial frequency and contrast at discrete intervals, e.g., along a plurality of sweep trajectories, e.g., where the sweeps all begin with the target at a common origin that then radiate outward along vectors in contrast sensitivity function (CSF) space until they reach the limit of function, at which point target invisibility prevents further tracking by the subject and a threshold is recorded);
automatically perform, in real time, during the course of the functional vision test, a visibility determination using a machine learning algorithm to automatically identify if the subject is accurately tracking the target in time and space; and
compute and/or display test results using the visibility determination.
89 . The system of claim 88 , wherein the test results comprise a plot of contrast sensitivity (e.g., inverse of root-mean-square (RMS) contrast ratio) and spatial frequency (in cycles per degree, CPD).
90 . The system of claim 88 or 89 , wherein the machine learning algorithm (e.g., said machine learning algorithm comprising one or more recombinant neural networks, and/or long short-time memory, and/or one or more temporal convolutional networks) has been previously trained from user trials with ground truth established by manual assessment of video recordings of the users that determined if a subject eye position was within an individual target (e.g., culled from hundreds of minutes of observation and annotation).
91 . The system of any one of claims 88-90 , wherein the machine learning algorithm determines, during the course of the functional vision test, a probability that, for a given time window (e.g., in milliseconds, e.g., <250 milliseconds, <100 milliseconds, <50 milliseconds, <25 milliseconds, etc.) the subject is (or is not) observing the target (e.g., an individual target in a field of a plurality of targets, e.g., 3 or more targets, e.g., 5 targets), wherein appearance of the target as presented to the subject is altered (e.g., value of contrast and/or value of spatial frequency) at least once during the course of the functional vision test upon determination the subject is (likely) observing the target (e.g., wherein the appearance of the target stops being altered when the algorithm determines the subject is no longer tracking the target, wherein final values of contrast and spatial frequency are recorded as the subject's visual/functional threshold for that parameter space, e.g., at which point a new target is presented and the process is repeated, e.g., a new sweep is conducted).
92 . The system of any one of claims 88-91 , wherein the instructions, when executed by the processor, cause the processor to adjust one or more threshold parameters (e.g., during the course of the functional vision test) (e.g., wherein the one or more threshold parameters comprises one or both of (i) and (ii) as follows: (i) a tolerance for how near or far the subject's actual eye position is located from the center of the target, e.g., the circular target, and (ii) a time window for the visibility determination using the machine learning algorithm.
93 . A method for conducting a functional vision test (e.g., a contrast sensitivity and/or spatial frequency test, e.g., a radial sweep test) on a subject using a virtual- and/or augmented- and/or mixed-reality device (e.g., VR headset with eye tracking capability), the method comprising:
rendering and displaying (e.g., via the VR headset) to the subject, by a processor of a computing device, over a course of the functional vision test, a target (e.g., a moving target, e.g., a target that changes in spatial frequency and contrast at discrete intervals, e.g., along a plurality of sweep trajectories, e.g., where the sweeps all begin with the target at a common origin that then radiate outward along vectors in contrast sensitivity function (CSF) space until they reach the limit of function, at which point target invisibility prevents further tracking by the subject and a threshold is recorded);
automatically performing, by the processor, in real time, during the course of the functional vision test, a visibility determination using a machine learning algorithm to automatically identify if the subject is accurately tracking the target in time and space; and
computing and/or displaying, by the processor, test results using the visibility determination.
94 . The method of claim 93 , wherein the test results comprise a plot of contrast sensitivity (e.g., inverse of root-mean-square (RMS) contrast ratio) and spatial frequency (in cycles per degree, CPD).
95 . The method of claim 93 or 94 , wherein the machine learning algorithm (e.g., said machine learning algorithm comprising one or more recombinant neural networks, and/or long short-time memory, and/or one or more temporal convolutional networks) has been previously trained from user trials with ground truth established by manual assessment of video recordings of the users that determined if a subject eye position was within an individual target (e.g., culled from hundreds of minutes of observation and annotation).
96 . The method of any one of claims 93-95 , wherein the machine learning algorithm determines, during the course of the functional vision test, a probability that, for a given time window (e.g., in milliseconds, e.g., <250 milliseconds, <100 milliseconds, <50 milliseconds, <25 milliseconds, etc.) the subject is (or is not) observing the target (e.g., an individual target in a field of a plurality of targets, e.g., 3 or more targets, e.g., 5 targets), wherein appearance of the target as presented to the subject is altered (e.g., value of contrast and/or value of spatial frequency) at least once during the course of the functional vision test upon determination the subject is (likely) observing the target (e.g., wherein the appearance of the target stops being altered when the algorithm determines the subject is no longer tracking the target, wherein final values of contrast and spatial frequency are recorded as the subject's visual/functional threshold for that parameter space, e.g., at which point a new target is presented and the process is repeated, e.g., a new sweep is conducted).
97 . The method of any one of claims 93-96 , comprising adjusting, by the processor, one or more threshold parameters (e.g., during the course of the functional vision test) (e.g., wherein the one or more threshold parameters comprises one or both of (i) and (ii) as follows: (i) a tolerance for how near or far the subject's actual eye position is located from the center of the target, e.g., the circular target, and (ii) a time window for the visibility determination using the machine learning algorithm.
98 . The method of any one of claims 93-97 , further comprising performing steps of any one of claims 77-87 .
99 . The method of any one of claims 1-26 , further comprising performing steps of any one of claims 77-87 .
100 . A method of determining therapeutic effectiveness (e.g., benefit) of a therapeutic intervention, the method comprising (i) administering a functional vision test according to the method of any one of claims 1-26 and (ii) determining therapeutic effectiveness (e.g., benefit) of a therapeutic intervention (e.g., laser coagulation therapy) to a subject based on test results from the functional vision test.