Method and device for controlling a device for aiding vision
An input signal (f) representative of a scene to be viewed comprises, for each pixel of a matrix of pixels, an event-based asynchronous-signal sequence obtained as a function of variations of light relating to the pixel in the scene. This input signal is transformed spatially within the matrix of pixels and temporally along the signal sequences to generate respective control signals (S) for pixel zones of a device for aiding vision ( 20 ).
1. A method of controlling a visual aid device, comprising:
receiving an input signal representative of a scene to be viewed, the input signal comprising, for each pixel in a matrix of pixels, an event-based asynchronous signal sequence obtained as a function of variations of light relating to the pixel in the scene;
transforming the input signal spatially within the matrix of pixels and temporally along the event-based asynchronous signal sequences to generate respective control signals for pixel zones of the visual aid device; and
applying the control signals to the visual aid device,
wherein the event-based asynchronous signal sequence for a pixel comprises a sequence of pulses temporally positioned as a function of the light variations relating to the pixel,
wherein transforming the input signal to generate the control signals comprises:
obtaining a first signal resulting from two spatial filtering operations with filtering kernels of different sizes, calculation of a difference between results of the two spatial filtering operations, and a temporal filtering operation on the difference; and
obtaining a second signal of zero value if the first signal has a value of a specific sign, and of the same absolute value as the first signal otherwise.
2. The method of claim 1 , wherein the vision aid device comprises a subretinal visual prosthesis, and the control signals applied to the visual prosthesis are generated from said second signal.
3. The method of claim 1 , wherein the visual aid device comprises a matrix of light-emitting elements and the control signals applied to said matrix are generated from said second signal.
4. The method of claim 1 , wherein at least a first excitatory signal and a first inhibitory signal are obtained with respective time constants for the temporal filtering operation on the difference, and wherein at least a second excitatory signal and a second inhibitory signal are respectively obtained from the first excitatory and inhibitory signals.
5. The method of claim 4 , wherein transforming the input signal to generate the control signals further comprises:
obtaining a third signal resulting from a spatial filtering operation on the difference between the second excitatory signal and an inhibitory component derived from the second inhibitory signal; and
when the third signal for a given pixel zone of the visual aid device exceeds a predetermined threshold value, inserting a pulse into the control signal intended for said pixel zone and resetting the third signal for said pixel zone to zero.
6. The method of claim 5 , wherein derivation of the inhibitory component from the second inhibitory signal comprises the application of a predetermined delay and a spatial filtering operation.
7. The method of claim 6 , wherein the spatial filtering operation in the derivation of the inhibitory component uses an off-center filtering kernel.
8. The method of claim 4 , wherein the second excitatory and inhibitory signals are obtained for a first channel and for a second channel with temporal filtering operations at respective time constants, and wherein transforming the input signal to generate the control signals further comprises:
obtaining a third signal resulting from a spatial filtering operation on the difference between a linear combination of the second excitatory signals for the first and second channels and an inhibitory component derived from the second inhibitory signals for the first and second channels; and
when the third signal for a given pixel zone of the visual aid device exceeds a given threshold value, inserting a pulse into the control signal for said pixel zone and resetting the third signal for said pixel zone to zero.
9. The method of claim 8 , wherein derivation of the inhibitory component from the second inhibitory signals comprises the application of respective delays to the second inhibitory signals for the first and second channels, a spatial filtering operation on the delayed second inhibitory signals, and calculation of a linear combination of delayed and filtered second inhibitory signals.
10. The method of claim 5 , wherein the visual aid device comprises a visual prosthesis in an epiretinal or cortical position or on the lateral geniculate body, and the control signals applied to the visual prosthesis are generated from said third signal.
11. The method of claim 5 , wherein the visual aid device includes a matrix of light-emitting elements and the control signals applied to this matrix are generated from said third signal.
12. The method of claim 1 , wherein transforming the input signal in order to generate the control signals is carried out using an artificial neural network.
13. A device for processing signals for controlling a visual aid device, comprising:
an input for receiving an input signal representative of a scene to be viewed, the input signal comprising, for each pixel in a matrix of pixels, an event-based asynchronous signal sequence obtained as a function of variations of light relating to the pixel in the scene;
an output for supplying the control signals for the visual aid device; and
a processing circuit for generating the control signals, respective control signals being generated for pixel zones of the visual aid device by transforming the input signal spatially within the matrix of pixels and temporally along the event-based asynchronous signal sequences,
wherein the event-based asynchronous signal sequence for a pixel comprises a sequence of pulses temporally positioned as a function of the light variations relating to the pixel,
wherein the processing circuit is configured for transforming the input signal by:
obtaining a first signal resulting from two spatial filtering operations with filtering kernels of different sizes, calculation of a difference between results of the two spatial filtering operations, and a temporal filtering operation on the difference; and
obtaining a second signal of zero value if the first signal has a value of a specific sign, and of the same absolute value as the first signal otherwise.
14. The device of claim 13 , wherein at least a first excitatory signal and a first inhibitory signal are obtained with respective time constants for the temporal filtering operation on the difference, and wherein at least a second excitatory signal and a second inhibitory signal are respectively obtained from the first excitatory and inhibitory signals.
15. The device of claim 14 , wherein the processing circuit is further configured for transforming the input signal by:
obtaining a third signal resulting from a spatial filtering operation on the difference between the second excitatory signal and an inhibitory component derived from the second inhibitory signal; and
when the third signal for a given pixel zone of the visual aid device exceeds a predetermined threshold value, inserting a pulse into the control signal intended for said pixel zone and resetting the third signal for said pixel zone to zero.
16. The device of claim 14 , wherein the second excitatory and inhibitory signals are obtained for a first channel and for a second channel with temporal filtering operations at respective time constants, and wherein the processing circuit is further configured for transforming the input signal by:
obtaining a third signal resulting from a spatial filtering operation on the difference between a linear combination of the second excitatory signals for the first and second channels and an inhibitory component derived from the second inhibitory signals for the first and second channels; and
when the third signal for a given pixel zone of the visual aid device exceeds a given threshold value, inserting a pulse into the control signal for said pixel zone and resetting the third signal for said pixel zone to zero.
17. The device of claim 13 , wherein the processing circuit comprises an artificial neural network.
18. The method of claim 8 , wherein the visual aid device comprises a visual prosthesis in an epiretinal or cortical position or on the lateral geniculate body, and the control signals applied to the visual prosthesis are generated from said third signal.
19. The method of claim 8 , wherein the visual aid device includes a matrix of light-emitting elements and the control signals applied to this matrix are generated from said third signal.