Granted Patent B2

US 12,423,567 · App. 17/485,073 · Granted Sep 23, 2025

Inventors: Swagath Venkataramani (White Plains, NY); Shubham Jain (Elmsford, NY); Leland Chang (New York, NY)

Assignee: International Business Machines Corporation

G06N3/065G06F17/16G06N3/084G06G7/16

Abstract

Claims (51)

1. A system, comprising:

an analog resistive processing unit system comprising at least one array of resistive processing unit cells; and

one or more processors configured to:

configure the analog resistive processing unit system to implement a convolutional neural network comprising a convolutional layer, wherein the convolutional layer comprises at least one kernel matrix;

program the at least one array of resistive processing unit cells to store a transformed kernel matrix, wherein the transformed kernel matrix is generated by applying a first transformation process to the at least one kernel matrix using a first predefined transformation matrix; and

utilize the analog resistive processing unit system to perform an analog convolution operation by performing analog matrix-vector multiplication operations using the transformed kernel matrix and input vectors of a transformed data matrix, to thereby generate a transformed convolution output matrix, wherein the transformed data matrix is generated by applying a second transformation process to a data matrix using a second predefined transformation matrix;

wherein the one or more processors are configured to generate the transformed data matrix by utilizing the analog resistive processing unit system to perform the second transformation process in an analog domain.

2. The system of claim 1 , wherein the analog convolution operation is performed as part of a model training process to train the convolutional neural network implemented on the analog resistive processing unit system.

3. The system of claim 1 , wherein the analog convolution operation is performed as part of an inference process that is performed using the convolutional neural network implemented on the analog resistive processing unit system.

4. The system of claim 1 , wherein the first and second transformation processes and the analog convolution operation are implemented according to a Winograd filtering function.

5. The system of claim 1 , wherein the one or more processors are configured to compute the transformed kernel matrix in a digital domain by a process which comprises multiplying the first predefined transformation matrix and the at least one kernel matrix to generate an intermediate matrix, and multiplying the intermediate matrix and a transpose of first predefined transformation matrix to thereby generate the transformed kernel matrix.

6. The system of claim 1 , wherein the convolutional layer comprises a plurality of kernel matrices, and wherein the one or more processors are configured to:

generate a corresponding transformed kernel matrix for each kernel matrix of the plurality of kernel matrices; and

store each transformed kernel matrix in a separately addressable region of the at least one array of resistive processing unit cells.

7. The system of claim 6 , wherein in utilizing the analog resistive processing unit system to perform the analog convolution operation, the one or more processors are configured perform a pipeline parallel process by applying each input vector of the transformed data matrix to a corresponding one of the transformed kernel matrices stored in the separately addressable regions of the at least one array of resistive processing unit cells.

8. The system of claim 1 , wherein in performing the second transformation process in the analog domain, the one or more processors are configured to:

program the at least one array of resistive processing unit cells to store the second predefined transformation matrix; and

perform analog matrix-vector multiplication operations by inputting vectors of the data matrix to the stored second predefined transformation matrix.

9. The system of claim 1 , wherein in performing the second transformation process in the analog domain, the one or more processors are configured to:

program the at least one array of resistive processing unit cells to store the second predefined transformation matrix;

program the at least one array of resistive processing unit cells to store a transpose of the second predefined transformation matrix;

perform analog matrix-vector multiplication operations by sequentially inputting vectors of the data matrix to the stored second predefined transformation matrix to generate a corresponding sequence of intermediate output vectors; and

perform analog matrix-vector multiplication operations by inputting the sequence of intermediate output vectors to the stored transpose of the second predefined transformation matrix to thereby generate plurality of output vectors which are combined to generate the transformed data matrix.

10. The system of claim 1 , wherein the one or more processors are configured to utilize the analog resistive processing unit system to perform an analog inverse transformation process using a third transformation matrix stored in the at least one array of resistive processing unit cells to thereby convert the transformed convolution output matrix to a convolution output matrix in a spatial domain.

11. The system of claim 1 , wherein the one or more processors are configured to utilize the analog resistive processing unit system to generate a transformed error gradient matrix which is backpropagated through layers of the convolutional neural network implemented on the analog resistive processing unit system, wherein the transformed error gradient matrix is utilized to update transformed kernel weight of the transformed kernel matrix stored in the at least one array of resistive processing unit cells.

12. A computer program product, comprising:

one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions comprising:

program instructions to configure an analog resistive processing unit system to implement a convolutional neural network comprising a convolutional layer, wherein the convolutional layer comprises at least one kernel matrix, wherein the analog resistive processing unit system comprises at least one array of resistive processing unit cells; and

program instructions to program the at least one array of resistive processing unit cells to store a transformed kernel matrix, wherein the transformed kernel matrix is generated by applying a first transformation process to the at least one kernel matrix using a first predefined transformation matrix;

program instructions to utilize the analog resistive processing unit system to perform an analog convolution operation by performing analog matrix-vector multiplication operations using the transformed kernel matrix and input vectors of a transformed data matrix, to thereby generate a transformed convolution output matrix, wherein the transformed data matrix is generated by applying a second transformation process to a data matrix using a second predefined transformation matrix; and

program instructions to generate the transformed data matrix by utilizing the analog resistive processing unit system to perform the second transformation process in an analog domain.

13. The computer program product of claim 12 , wherein the first and second transformation processes and the analog convolution operation are implemented according to a Winograd filtering function.

14. The computer program product of claim 12 , wherein the convolutional layer comprises a plurality of kernel matrices, and further comprising:

program instructions to generate a corresponding transformed kernel matrix for each kernel matrix of the plurality of kernel matrices; and

program instructions to store each transformed kernel matrix in a separately addressable region of the at least one array of resistive processing unit cells.

15. The computer program product of claim 14 , wherein the program instructions to utilize the analog resistive processing unit system to perform the analog convolution operation, comprise program instructions to perform a pipeline parallel process by applying each input vector of the transformed data matrix to a corresponding one of the transformed kernel matrices stored in the separately addressable regions of the at least one array of resistive processing unit cells.

16. The computer program product of claim 12 , wherein the program instructions to generate the transformed data matrix by utilizing the analog resistive processing unit system to perform the second transformation process in the analog domain, comprise:

program instructions to program the at least one array of resistive processing unit cells to store the second predefined transformation matrix; and

program instructions to perform analog matrix-vector multiplication operations by inputting vectors of the data matrix to the stored second predefined transformation matrix.

17. The computer program product of claim 12 , further comprising program instructions to utilize the analog resistive processing unit system to perform an analog inverse transformation process using a third transformation matrix stored in the at least one array of resistive processing unit cells to thereby convert the transformed convolution output matrix to a convolution output matrix in a spatial domain.

18. A neuromorphic computing system, comprising:

an analog resistive processing unit system comprising at least one array of resistive processing unit cells;

wherein the analog resistive processing unit system implements a convolutional neural network comprising a convolutional layer, wherein the convolutional layer comprises at least one kernel matrix;

wherein the at least one array of resistive processing unit cells stores a transformed kernel matrix, wherein the transformed kernel matrix is generated by applying a first transformation process to the at least one kernel matrix using a first predefined transformation matrix;

wherein the analog resistive processing unit system is configured to perform an analog convolution operation by performing analog matrix-vector multiplication operations using the transformed kernel matrix and input vectors of a transformed data matrix, to thereby generate a transformed convolution output matrix, wherein the transformed data matrix is generated by applying a second transformation process to a data matrix using a second predefined transformation matrix; and

wherein the at least one array of resistive processing unit cells stores the second predefined transformation matrix, and the transformed data matrix is generated by utilizing the analog resistive processing unit system performing the second transformation process in an analog domain.

19. The neuromorphic computing system of claim 18 , wherein the second transformation process is performed in the analog domain by performing analog matrix-vector multiplication operations by inputting vectors of the data matrix to the stored second predefined transformation matrix.

20. The neuromorphic computing system of claim 18 , wherein the second transformation process is performed in the analog domain by:

storing a transpose of the second predefined transformation matrix in the at least one array of resistive processing unit cells;

performing analog matrix-vector multiplication operations by sequentially inputting vectors of the data matrix to the stored second predefined transformation matrix to generate a corresponding sequence of intermediate output vectors; and

performing analog matrix-vector multiplication operations by inputting the sequence of intermediate output vectors to the stored transpose of the second predefined transformation matrix to thereby generate plurality of output vectors which are combined to generate the transformed data matrix.

Assignments (1)

Continuity (1)

References Cited (27)