Frame image conversion spike system
A frame image conversion spike system is disclosed. In order to solve the technical problem of high power consumption when converting a frame image into a spike event, a frame rate or difference frame frequency is adaptively adjusted according to the amount of effective pixel change between frame images, so as to reduce system power consumption and adaptively respond to scene application needs. In order to solve the issue of low inference accuracy after converting frame images into spike events and directly applying them to an SNN processor, a global randomization scheme is proposed considering the randomization process of images as a whole. It has opened up all the links of applying a frame image sensor to the SNN processor and provided an alternative to an event camera. This is suitable for a brain-inspired chip and AIoT field.
1 . A spike train randomization method, comprising:
obtaining a pixel value of a difference frame between frame images;
calculating a number of spike events corresponding to the difference frame pixel value according to the difference frame pixel value;
wherein the spike train randomization method further comprises at least one of the following steps:
i) storing a pixel coordinate corresponding to the difference frame pixel value in the same number of storage units as the number of spike events; reading the pixel coordinate stored in the storage unit corresponding to a random number according to a currently obtained random number; generating spike events in a target spike train at least according to a read pixel coordinate information;
ii) storing the number of spike events in the storage unit corresponding to the pixel coordinate of the difference frame pixel value; reading the number of spike events stored in the storage unit corresponding to the random number according to the currently obtained random number; if a read number of spike events is non-zero, generating spike events in the target spike train at least according to a pixel coordinate information of the difference frame pixel value corresponding to the storage unit.
2 . The spike train randomization method of claim 1 , wherein:
the difference frame pixel value is a difference between pixel values between two adjacent frames of images, or an absolute value of the difference between the pixel values between two adjacent frames of images.
3 . The spike train randomization method of claim 1 , wherein:
calculating the number of spike events corresponding to the difference frame pixel value is obtained according to one of the following methods:
i) a ratio of a product of the difference frame pixel value and a maximum number of spike events allowed to be issued by a single pixel to a theoretical maximum value of the pixel in the difference frame;
ii) a ratio of the product of the difference frame pixel value and the maximum number of spike events allowed to be issued by a single pixel to a maximum pixel value in the difference frame;
iii) a ratio of the product of the difference frame pixel value and the maximum number of spike events allowed to be issued by a single pixel to a larger one of the maximum pixel value in the difference frame or a first lower limit value.
4 . The spike train randomization method of claim 1 , wherein:
after obtaining the difference frame pixel value between the frame images, if the difference frame pixel value is smaller than a second lower limit value, the difference frame pixel value is set to zero.
5 . The spike train randomization method of claim 1 , wherein:
a random number sequence is generated by a random number generation module, and a currently obtained random number comes from the random number sequence;
the random number sequence does not generate the same random number within a cycles.
6 . The spike train randomization method of claim 5 , wherein:
the random number generation module comprises at least one random number generator based on a linear feedback shift register; or a period of the random number sequence is equal to a product of a number of pixels in the difference frame and a maximum number of spike events allowed to be issued by a single pixel, or equal to the number of pixels in the difference frame.
7 . The spike train randomization method of claim 1 , wherein:
according to the currently obtained random number, when reading the number of spike events stored in the storage unit corresponding to the random number, the random number comes from a random number sequence, and a period of the random number sequence is equal to the number of pixels in the difference frame;
for each difference frame, the random number sequence is executed for a number of cycles equal to a maximum number of spike events allowed to be issued by a single pixel.
8 . The spike train randomization method of claim 1 , wherein:
after acquiring difference frame pixel values between the frame images, storing the difference frame pixel values in a difference frame storage space; or,
after acquiring the difference frame pixel values between the frame images, the number of spike events corresponding to the difference frame pixel value frames is directly calculated without storing the difference frame pixel values.
9 . The spike train randomization method of claim 1 , wherein:
a number of storage units storing the pixel coordinates corresponding to the pixel values of the difference frame is less than a product of the number of pixels in the difference frame and a maximum number of spike events allowed to be issued by a single pixel.
10 . The spike train randomization method of claim 1 , wherein:
a number of storage units storing the pixel coordinates corresponding to the pixel values of the difference frame is less than a product of the number of pixels in the difference frame and a maximum number of spike events allowed to be issued by a single pixel.
11 . The spike train randomization method of claim 1 , wherein:
if the number of spike events read is not zero, the number of spike events stored in the storage unit corresponding to the random number is reduced by one.