Retina prosthesis
The invention provides a retinal prosthetic method and device that mimics the responses of the retina to a broad range of stimuli, including natural stimuli. Ganglion cell firing patterns are generated in response to a stimulus using a set of encoders, interfaces, and transducers, where each transducer targets a single cell or a small number of cells. The conversion occurs on the same time scale as that carried out by the normal retina. In addition, aspects of the invention may be used with robotic or other mechanical devices, where processing of visual information is required. The encoders may be adjusted over time with aging or the progression of a disease.
1. A method comprising:
receiving, by a processing device, raw image data corresponding to a series of raw images; and
processing the raw image data with an encoder of the processing device to generate encoded data, wherein the encoder is characterized by an input/output transformation that substantially mimics the input/output transformation of at least one retinal cell a vertebrate retina;
wherein the processing the raw image data comprises applying a spatiotemporal transformation to a set of images to generate a set of retinal response values, and wherein the applying the spatiotemporal transformation further comprises:
applying a spatiotemporal kernel to the set of images to generate spatially-temporally transformed images; and
applying a nonlinear function to the spatially-temporally transformed images to generate the set of retinal response values;
generating the encoded data based on the set of retinal response values; and
activating a high resolution signaling device according to the encoded data.
2. The method of claim 1 , wherein processing the raw image data with an encoder to generate encoded data comprises:
processing the raw image data to generate a plurality of values, X,
transforming the plurality of X values into a plurality of response values, λ m , indicative of a corresponding response of a retinal cell in the retina, m, and generating the encoded data based on the response values.
3. The method of claim 2 , wherein the response values correspond to retinal cell firing rates.
4. The method of claim 3 , wherein the response values correspond to a function of the retinal cell firing rates.
5. The method of claim 3 , wherein the response values correspond to retinal cell output pulses.
6. The method of claim 3 , wherein the response values correspond to retinal cell generator potential.
7. The method of claim 1 , wherein processing the raw image data with an encoder to generate encoded data comprises:
receiving images from the raw image data and, for each image, rescaling the luminance or contrast to generate a rescaled image stream;
receiving a set of N rescaled images from the rescaled image stream and applying the spatiotemporal transformation to the set of N images to generate the set of retinal response values, each value in the set corresponding to a respective one of the retinal cells.
8. The method of claim 7 , wherein the response values comprise retina cell firing rates.
9. The method of claim 7 wherein N is at least 5.
10. The method of claim 7 wherein N is at least 20.
11. The method of claim 7 , wherein applying the spatiotemporal kernel comprises:
convolving of the N rescaled images with the spatiotemporal kernel to generate N spatially-temporally transformed images.
12. The method of claim 11 , wherein applying the spatiotemporal kernel comprises:
convolving the N rescaled images with a spatial kernel to generate N spatially transformed images;
convolving the N spatially transformed images with a temporal kernel to generate a temporal transformation output; and
applying a nonlinear function to the temporal transformation output to generate the set of retinal response values.
13. The method of claim 1 , wherein the encoder is characterized by a set of parameters, and wherein the values of the parameters are determined using response data obtained experimentally from a mammalian retina while said retina is exposed to white noise and natural scene stimuli.
14. The method of claim 13 , wherein the encoder is configured such that the Pearson's correlation coefficient between a test input stimulus and a corresponding stimulus reconstructed from the encoded data that would be generated by the encoder in response to the test input stimulus is at least about 0.35.
15. The method of claim 13 , wherein the encoder is configured such that the Pearson's correlation coefficient between a test input stimulus and a corresponding stimulus reconstructed from the encoded data that would be generated by the encoder in response to the test input stimulus is at least about 0.65.
16. The method of claim 13 , wherein the encoder is configured such that the Pearson's correlation coefficient between a test input stimulus and a corresponding stimulus reconstructed from the encoded data that would be generated by the encoder in response to the test input stimulus is at least about 0.95.
17. The method of claim 13 , wherein the test input stimulus comprises a series of natural scenes.