Spiking neural network arrangement for active SPAD imaging
View Patent ↗A spiking neural network system for single-photon imaging is disclosed. The system comprises: a light source for emitting a series of light pulses, one pulse per repetition period, for repeatedly illuminating one or more objects; a single-photon detector for detecting photons received from the one or more objects; a ring circuit connected to the single-photon detector, the ring circuit comprising a set of delay elements connected to one another thereby forming a ring structure, a respective delay element is configured to store a spike received from the single-photon detector, the ring circuit being configured to repeatedly rotate the spikes through the delay elements, a respective spike making a full rotation of the ring circuit in one rotation period such that the ring circuit is synchronized with the light source; a read-out circuit for reading out the spikes stored in the delay elements while maintaining the differences of arrival times of the spikes in the ring circuit; and a spiking neural network configured to receive the spikes from the read-out circuit to generate an image of the one or more objects.
1 . A spiking neural network system for single-photon imaging, the system comprising:
a light source for emitting a series of light pulses, one pulse per repetition period, for repeatedly illuminating one or more objects;
a single-photon detector for detecting photons received from the one or more objects;
a ring circuit directly or indirectly connected to the single-photon detector, the ring circuit comprising a set of delay elements connected to one another by a connector arrangement thereby forming a ring structure, a respective delay element is configured to store a data representation of a respective detected photon, referred to as a spike, received from the single-photon detector, the ring circuit being configured to repeatedly rotate the spikes through the delay elements of the ring circuit, a respective spike making a full rotation of the ring circuit in one rotation period such that the ring circuit is synchronized with the light source;
a read-out circuit for reading out the spikes stored in the delay elements while maintaining the differences of arrival times of the spikes in the ring circuit; and
a spiking neural network configured to receive the spikes from the read-out circuit to generate an image of the one or more objects as a response to receiving the spikes forming a spike train from the read-out circuit.
2 . The system according to claim 1 , wherein the single-photon detector is a single-photon avalanche diode.
3 . The system according to claim 1 , wherein the read-out circuit is configured to sequentially read out the spikes stored in the delay elements.
4 . The system according to claim 1 , wherein the read-out circuit is configured to read out the spikes only after a given number of photons have been detected by the single-photon detector or after a given time has elapsed since turning on the light source or since the last read-out operation.
5 . The system according to claim 1 , wherein the ring circuit is a ring oscillator, a dynamic random-access memory, or a flip-flop circuit arrangement, and wherein the flip-flop circuit arrangement optionally comprises a series of D-type flip-flop circuits in a ring structure.
6 . The system according to claim 1 , wherein the ring circuit is synchronized with the light source such that the length of the repetition period equals or substantially equals the length of the rotation period.
7 . The system according to claim 1 , wherein the ring circuit is configured to receive a synchronization signal from the light source, or the light source is configured to receive a synchronization signal from the ring circuit to keep the light source and the ring circuit mutually synchronized, or the light source and the ring circuit are configured to receive an external synchronization signal.
8 . The system according to claim 1 , wherein the system further comprises an injection device placed between the single-photon detector and the ring circuit for injecting the spikes generated by the single-photon detector in response to photon detections into the ring circuit.
9 . The system according to claim 1 , wherein the number of delay elements in the ring circuit is comprised between 50 and 10000, and more specifically between 100 and 1000.
10 . The system according to any claim 1 , wherein the spiking neural network comprises a given number of hidden layers, and only one input neuron node and only one output neuron node, or wherein the spiking neural network comprises a given number of hidden layers, and at least two one input neuron nodes and/or at least two output neuron nodes, or wherein the spiking neural network is a convolutional neural network or a recurrent neural network.
11 . The system according to claim 10 , wherein the number of hidden layers in the spiking neural network is comprised between 1 and 20, and more specifically between 1 and 10.
12 . The system according to claim 10 , wherein the number of artificial neurons per hidden layer decreases toward the output neuron node.
13 . The system according to claim 1 , wherein the spiking neural network adopts an integrate-and-fire model, a leaky integrate-and-fire model, adaptive integrate-and-fire model, or a Hodgkin-Huxley model.
14 . The system according to claim 1 , wherein a respective delay element is configured to store one spike only as a logic low or logic high.
15 . The system according to claim 1 , wherein the repetition period is comprised between 0.01 μs and 100 μs, and more specifically between 0.1 μs and 10 μs.
16 . The system according to claim 1 , wherein the one or more objects are arranged to undergo a dynamic process.
17 . The system according to claim 1 , wherein the spiking neural network is trainable with surrogate gradient.
18 . An imaging method for generating an image of one or more objects by using a spiking neural network system for single-photon imaging, the method comprising:
emitting by using a light source a series of light pulses, one pulse per repetition period, for repeatedly illuminating one or more objects;
detecting by a single-photon detector photons received from the one or more objects;
feeding by the single-photon detector or by an injection circuit the detected photons to a ring circuit directly or indirectly connected to the single-photon detector, the ring circuit comprising a set of delay elements connected to one another by a connector arrangement thereby forming a ring structure, a respective delay element is configured to store a data representation of a respective detected photon, referred to as a spike, received from the single-photon detector;
repeatedly rotating by the ring circuit the spikes through the delay elements of the ring circuit, a respective spike making a full rotation of the ring circuit in one rotation period such that the ring circuit is synchronized with the light source;
reading out by a read-out circuit the spikes stored in the delay elements while maintaining the differences of arrival times of the spikes in the ring circuit; and
receiving by a spiking neural network the spikes from the read-out circuit, and generating an image of the one or more objects as a response to receiving the spikes from the read-out circuit.
19 . The method according to claim 18 , wherein the method further comprises initializing the delay elements once the delay elements have been read out.
20 . The method according to claim 18 , wherein the method further comprises, prior to operating the spiking neural network to generate the image, training the spiking neural network with surrogate gradient.