Vision defect determination and enhancement using a prediction model
In certain embodiments, enhancement of a vision of a user may be facilitated. In some embodiments, a visual test presentation including a set of stimuli may be provided to a user. Feedback related to the set of stimuli may be obtained during the visual test presentation. The feedback may be provided to a prediction model, and the prediction model may be configured based on the feedback. Subsequent to the configuring of the prediction model, live video stream data, eye characteristic information, or other information may be provided to the prediction model to obtain a set of modification parameters. A modified version of the live video stream data may be generated and displayed to the user based on the live video stream and the set of modification parameters.
1. A system for facilitating enhancement related to a vision of a user, the system comprising:
a computer system that comprises one or more processors executing computer program instructions that, when executed, cause the computer system to:
obtain feedback related to a set of stimuli displayed to a user, the feedback related to the set of stimuli indicating whether or how the user sees one or more stimuli of the set;
provide the feedback related to the set of stimuli to a neural network to train the neural network, the neural network being updated based on the feedback related to the set of stimuli;
provide, via a wearable device, live video stream data or eye characteristic information to the neural network to obtain outputs from the neural network, the eye characteristic information indicating one or more characteristics of one or more eyes of the user that occurred during a live capture of the live video stream data; and
cause, via the wearable device, based on the neural network outputs, a modified version of the live video stream data to be displayed to the user within less than one second of the live video stream data being captured via one or more cameras.
2. The system of claim 1 , wherein obtaining the feedback related to the set of stimuli comprises obtaining an eye image captured during a visual test presentation, the eye image being an image of an eye of the user, and
wherein providing the feedback related to the set of stimuli comprises providing, during the visual test presentation, the eye image to the neural network to train the neural network.
3. The system of claim 2 , wherein the eye image is an image of a retina of the eye or an image of a cornea of the eye.
4. The system of claim 1 , wherein obtaining the feedback related to the set of stimuli comprises obtaining, via the wearable device, an indication of a response of the user to one or more stimuli of the set of stimuli or an indication of a lack of response of the user to one or more stimuli of the set of stimuli, and
wherein providing the feedback related to the set of stimuli comprises providing, via the wearable device during a visual test presentation, the indication of the response or the indication of the lack of response to the neural network to train the neural network.
5. The system of claim 4 , wherein the response comprises an eye movement, a gaze direction, or a pupil size change.
6. The system of claim 4 , wherein the response comprises a user modification of one or more stimuli via user input of the user.
7. The system of claim 1 , wherein obtaining the feedback related to the set of stimuli comprises obtaining an indication of a response of the user to one or more stimuli of the set of stimuli, and
wherein providing the feedback related to the set of stimuli comprises providing, during a visual test presentation, the indication of the response to the neural network to train the neural network.
8. The system of claim 1 , wherein obtaining the feedback related to the set of stimuli comprises obtaining an indication of a lack of response of the user to one or more stimuli of the set of stimuli, and
wherein providing the feedback related to the set of stimuli comprises providing, during a visual test presentation, the indication of the lack of response to the neural network to train the neural network.
9. The system of claim 1 , wherein the eye characteristic information indicates one or more gaze directions or pupil sizes of one or more eyes of the user that occurred during the live capture of the live video stream data.
10. The system of claim 1 , wherein the computer system is caused to:
monitor one or more changes related to one or more eyes of the user; and
cause, based on the monitoring, an adjustment of one or more shapes or sizes of first or second display portions on one or more transparent displays of the wearable device,
wherein causing the modified version of the live video stream data to be displayed comprises causing, based on the neural network outputs, the modified version of the live video stream data to be displayed on the first or second display portions.
11. The system of claim 1 , wherein the computer system is caused to:
obtain, as the neural network outputs, image data representing portions of the modified version of the live video stream data from the neural network based on the live video stream data being provided to the neural network.
12. The system of claim 1 , wherein the computer system is caused to:
obtain, as the neural network outputs, modification parameters from the neural network based on the live video stream data being provided to the neural network; and
generate the modified version of the live video stream data based on the live video stream data and the modification parameters.
13. A method being implemented by one or more processors executing computer program instructions that, when executed, perform the method, the method comprising:
obtaining feedback related to a set of stimuli displayed to a user;
providing the feedback related to the set of stimuli to a prediction model, the prediction model being configured based on the feedback related to the set of stimuli;
providing, via a wearable device, live video stream data or eye characteristic information to the prediction model to obtain outputs from the prediction model, the live video stream data representing an environment of the user, the eye characteristic information indicating one or more characteristics of one or more eyes of the user that occurred during a live capture of the live video stream data; and
causing, via the wearable device, based on the prediction model outputs, a modified version of the live video stream data to be displayed to the user.
14. The method of claim 13 , wherein obtaining the feedback related to the set of stimuli comprises obtaining an eye image captured during a visual test presentation, the eye image being an image of an eye of the user,
wherein providing the feedback related to the set of stimuli comprises providing, during the visual test presentation, the eye image to the prediction model, the prediction model being configured based on the eye image, and
wherein the eye image is an image of a retina of the eye or an image of a cornea of the eye.
15. The method of claim 13 , wherein obtaining the feedback related to the set of stimuli comprises obtaining, via the wearable device, an indication of a response of the user to one or more stimuli of the set of stimuli or an indication of a lack of response of the user to one or more stimuli of the set of stimuli, and
wherein providing the feedback related to the set of stimuli comprises providing, via the wearable device during a visual test presentation, the indication of the response or the indication of the lack of response to the prediction model, the prediction model being configured based on the indication of the response or the indication of the lack of response.
16. The method of claim 13 , further comprising:
obtaining, as the prediction model outputs, modification parameters from the prediction model based on the live video stream data being provided to the prediction model; and
generating the modified version of the live video stream data based on the live video stream data and the modification parameters.
17. The method of claim 13 , wherein the eye characteristic information indicates one or more gaze directions or pupil sizes of one or more eyes of the user that occurred during the live capture of the live video stream data.
18. One or more non-transitory computer-readable media comprising instructions that, when executed by one or more processors, cause operations comprising:
obtaining feedback related to a set of stimuli displayed to a user, the feedback related to the set of stimuli indicating whether or how the user sees one or more stimuli of the set;
providing the feedback related to the set of stimuli to a prediction model, the prediction model being configured based on the feedback related to the set of stimuli;
providing live video stream data or eye characteristic information to the prediction model to obtain outputs from the prediction model, the eye characteristic information indicating one or more gaze directions or pupil sizes of one or more eyes of the user that occurred during a live capture of the live video stream data; and
causing, based on the prediction model outputs, a modified version of the live video stream data to be displayed to the user.
19. The one or more non-transitory computer-readable media of claim 18 , wherein obtaining the feedback related to the set of stimuli comprises obtaining an eye image captured during a visual test presentation, the eye image being an image of an eye of the user,
wherein providing the feedback related to the set of stimuli comprises providing, during the visual test presentation, the eye image to the prediction model, the prediction model being configured based on the eye image, and
wherein the eye image is an image of a retina of the eye or an image of a cornea of the eye.
20. The one or more non-transitory computer-readable media of claim 18 , wherein obtaining the feedback related to the set of stimuli comprises obtaining, via a wearable device, an indication of a response of the user to one or more stimuli of the set of stimuli or an indication of a lack of response of the user to one or more stimuli of the set of stimuli, and
wherein providing the feedback related to the set of stimuli comprises providing, via the wearable device during a visual test presentation, the indication of the response or the indication of the lack of response to the prediction model, the prediction model being configured based on the indication of the response or the indication of the lack of response.
21. The one or more non-transitory computer-readable media of claim 18 , the operations further comprising:
obtaining, as the prediction model outputs, modification parameters from the prediction model based on the live video stream data or the eye characteristic information being provided to the prediction model; and
generating the modified version of the live video stream data based on the live video stream data and the modification parameters.
22. The one or more non-transitory computer-readable media of claim 18 , wherein obtaining the feedback related to the set of stimuli comprises obtaining, via a wearable device, an indication of a response of the user to one or more stimuli of the set of stimuli, the response comprising an eye movement, a gaze direction, a pupil size change, or a user modification of one or more stimuli via user input of the user, and
wherein providing the feedback related to the set of stimuli comprises providing, via the wearable device during a visual test presentation, the indication of the response to the prediction model, the prediction model being configured based on the indication of the response.