Batch size pipelined PIM accelerator for vision inference on multiple images

1. A method of pipelining inference of a neural network comprising a plurality of layers comprising an i-th layer (i being an integer greater than zero) and an (i+1)-th layer, the method comprising:

processing, for a first input image, by a controller using a composite filter comprising a first i-th filter associated with the i-th layer and a second i-th filter associated with the i-th layer, first i-th values of the i-th layer to generate first (i+1)-th values for the (i+1)-th layer, wherein the first i-th filter and the second i-th filter are offset in position by a stride length of the first i-th filter, such that a stride of the composite filter is greater than or equal to a sum of a stride of the first i-th filter and a stride of the second i-th filter, the stride length of the first i-th filter being associated with a movement of the first i-th filter across the i-th layer;

processing, for the first input image, the first (i+1)-th values of the (i+1)-th layer to generate output values; and

concurrently with processing, for the first input image, the (i+1)-th values,

processing, for a second input image, second i-th values of the i-th layer to generate second (i+1)-th values.

2. The method of claim 1 , wherein the processing, for the second input image, the second i-th values is performed concurrently with the processing, for the first input image, the first i-th values.

3. The method of claim 1 , wherein the first i-th values comprise pixel values of the first input image, and

wherein the second i-th values comprise pixel values of the second input image.

4. The method of claim 1 , wherein the first i-th values comprise values of a first feature map generated by a previous layer of the neural network, the first feature map corresponding to the first input image, and

wherein the second i-th values comprise values of a second feature map generated by the previous layer of the neural network, the second feature map corresponding to the second input image.

5. The method of claim 1 , wherein the processing, for the first input image, the first i-th values of the i-th layer comprises:

applying the composite filter to the first i-th values of the i-th layer to generate the (i+1)-th values for the (i+1)-th layer.

6. The method of claim 5 , wherein the processing, for the second input image, the second i-th values of the i-th layer comprises:

applying the composite filter to the second i-th values of the i-th layer to generate the second (i+1)-th values for the (i+1)-th layer.

7. The method of claim 5 , wherein the first i-th filter is a sliding convolutional filter in a form of a p×q matrix, where p and q are integers greater than zero.

8. The method of claim 5 , wherein applying the composite filter comprises:

performing a matrix multiplication operation between the composite filter and ones of the first i-th values overlapping the composite filter.

9. The method of claim 1 , wherein the processing, for the second input image, the second i-th values of the i-th layer is initiated a time offset after initiation of the processing, for the first input image, the first i-th values of the i-th layer, and

wherein the time offset is greater than or equal to a number of clock cycles corresponding to a single stride of the composite filter.

10. A system for pipelining inference of a neural network comprising a plurality of layers comprising an i-th layer (i being an integer greater than zero), an (i+1)-th layer, and an (i+2)-th layer, the system comprising:

a processor; and

a processor memory local to the processor, wherein the processor memory has stored thereon instructions that, when executed by the processor, cause the processor to perform:

processing, for a first input image, by a controller using a composite filter comprising a first i-th filter associated with the i-th layer and a second i-th filter associated with the i-th layer, first i-th values of the i-th layer to generate first (i+1)-th values for the (i+1)-th layer, wherein the first i-th filter and the second i-th filter are offset in position by a stride length of the first i-th filter, such that a stride of the composite filter is greater than or equal to a sum of a stride of the first i-th filter and a stride of the second i-th filter, the stride length of the first i-th filter being associated with a movement of the first i-th filter across the i-th layer;

processing, for the first input image, the first (i+1)-th values of the (i+1)-th layer to generate output values; and

concurrently with processing, for the first input image, the (i+1)-th values,

processing, for a second input image, second i-th values of the i-th layer to generate second (i+1)-th values.

11. The system of claim 10 , wherein the processing, for the second input image, the second i-th values is performed concurrently with the processing, for the first input image, the first i-th values.

12. The system of claim 10 , wherein the first i-th values comprise pixel values of the first input image, and

wherein the second i-th values comprise pixel values of the second input image.

13. The system of claim 10 , wherein the first i-th values comprise values of a first feature map generated by a previous layer of the neural network, the first feature map corresponding to the first input image, and

wherein the second i-th values comprise values of a second feature map generated by the previous layer of the neural network, the second feature map corresponding to the second input image.

14. The system of claim 10 , wherein the processing, for the first input image, the first i-th values of the i-th layer comprises:

applying the composite filter to the first i-th values of the i-th layer to generate the (i+1)-th values for the (i+1)-th layer.

15. The system of claim 14 , wherein the processing, for the second input image, the second i-th values of the i-th layer comprises:

applying the composite filter to the second i-th values of the i-th layer to generate the second (i+1)-th values for the (i+1)-th layer.

16. The system of claim 14 , wherein the first i-th filter is a sliding convolutional filter in a form of a p×q matrix, where p and q are integers greater than zero.

17. The system of claim 14 , wherein applying the composite filter comprises:

performing a matrix multiplication operation between the composite filter and ones of the first i-th values overlapping the composite filter.

18. The system of claim 10 , wherein the processing, for the second input image, the second i-th values of the i-th layer is initiated a time offset after initiation of the processing, for the first input image, the first i-th values of the i-th layer, and

wherein the time offset is greater than or equal to a number of clock cycles corresponding to a single stride of the composite filter.

19. A configurable processing-in-memory (PIM) system configured to implement a neural network, the system comprising:

a first at least one PIM subarray configured to perform a filtering operation of a composite filter comprising a first i-th filter and a second i-th filter of an i-th layer of the neural network (i being an integer greater than zero), wherein the first i-th filter and the second i-th filter are offset in position by a stride length of the first i-th filter, such that a stride of the composite filter is greater than or equal to a sum of a stride of the first i-th filter and a stride of the second i-th filter, the stride length of the first i-th filter being associated with a movement of the first i-th filter across the i-th layer;

a second at least one PIM subarray configured to perform a filtering operation of a (i+1)-th filter of an (i+1)-th layer of the neural network; and

a controller configured to control the first and second at least one PIM subarrays, the controller being configured to perform:

supplying first i-th values of the i-th layer to the first at least one PIM subarray to generate first (i+1)-th values for the (i+1)-th layer, the first i-th values corresponding to a first input image;

supplying the first (i+1)-th values of the (i+1)-th layer to the second at least one PIM subarray to generate output values associated with the first input image; and

concurrently with supplying the (i+1)-th values corresponding to the first input image,

supplying second i-th values of the i-th layer to the first at least one PIM subarray to generate second (i+1)-th values, the second i-th values corresponding to a second input image.

20. The system of claim 19 , wherein a PIM subarray of the first and second at least one PIM subarrays comprises:

a plurality of bitcells for storing a plurality of weights corresponding to a respective one of the first i-th or second i-th or (i+1)-th filters.