Granted Patent B2

US 12,314,844 · App. 17/340,242 · Granted May 27, 2025

Inventors: Tayfun Gokmen (Briarcliff Manor, NY); Wilfried Haensch (Somers, NY); Stefano Ambrogio (San Jose, CA); Charles Mackin (San Jose, CA)

Assignee: International Business Machines Corporation

G06N3/065G06N3/04G06N3/08G11C11/54G11C13/0021

Abstract

Claims (82)

1. A system, comprising:

a processor; and

a resistive processing unit coupled to the processor, the resistive processing unit comprising an array of cells, the cells respectively comprising resistive devices, at least a portion of the resistive devices being programmable to store weight values of a given matrix in the array of cells;

wherein, when the given matrix is stored in the array of cells, the processor is configured to perform a weight extraction process which comprises applying a set of input vectors to the resistive processing unit to perform analog matrix-vector multiplication operations on the given matrix, obtaining a set of output vectors resulting from the analog matrix-vector multiplication operations, and determining weight values of the given matrix stored in the array of cells utilizing the set of input vectors and the set of output vectors, the determining comprising:

generating a first matrix comprising the set of input vectors;

generating a second matrix comprising the set of output vectors; and

multiplying the first matrix by a transpose of the first matrix to thereby generate a third matrix.

2. The system of claim 1 , wherein in performing the weight extraction process, the processor is further configured to apply a set of linearly independent input vectors to the resistive processing unit.

3. The system of claim 1 , wherein:

the processor is configured to perform a model training process which comprises applying a set of training vectors to the resistive processing unit to perform analog matrix-vector multiplication operations on the given matrix during the model training process to thereby generate a trained model which comprises the stored weight values of the given matrix in the array of cells; and

in performing the weight extraction process, the processor is configured to determine the weight values of the given matrix stored in the array of cells utilizing the set of training vectors and a set of resulting vectors that are generated as a result of the analog matrix-vector multiplication operations performed during the model training process.

4. The system of claim 1 , wherein in determining the weight values of the given matrix stored in the array of cells, the processor is configured to perform a multivariate linear regression computation using the set of input vectors and the resulting set of output vectors to determine the weight values of the given matrix stored in the array of cells.

5. The system of claim 1 , wherein in determining the weight values of the given matrix stored in the array of cells, the processor is further configured to:

determine an inverse of the third matrix; and

multiply the inverse of the third matrix, the first matrix, and a transpose of the second matrix to thereby generate a fourth matrix;

wherein a transpose of the fourth matrix comprises the determined weight values of the given matrix stored in the array of cells.

6. The system of claim 1 , wherein the processor is further configured to:

compare the determined weight values with corresponding target weights of the given matrix stored in the array of cells to determine differences between the determined weight values and the corresponding target weight values; and

quantify one of a non-linear response and write noise associated with hardware elements of the resistive processing unit, based on the determined differences between the determined weight values and the corresponding target weight values.

7. The system of claim 1 , wherein the processor is further configured to:

perform a second weight extraction process to determine second weight values of the given matrix stored in the array of cells;

compare the determined weight values of the given matrix stored in the array of cells with corresponding ones of the second determined weight values of the given matrix stored in the array of cells to determine differences between the determined weight values and the corresponding second determined weight values; and

quantify one of a non-linear response and write noise associated with hardware elements of the resistive processing unit, based on the determined differences between the determined weight values and the corresponding second determined weight values.

8. The system of claim 1 , wherein the processor is further configured to:

receive an array of weights with target weight values;

store the array of weights in the array of cells of the resistive processing unit using programmed weight values which correspond to the target weight values;

estimate a transfer function which represents a relationship between the programmed weight values and the corresponding determined weight values obtained as a result of performing the weight extraction process; and

apply the target weight values to an inverse of the transfer function to thereby determine a new set of programmed weights.

9. The system of claim 8 , wherein the processor is further configured to:

determine whether the determined weight values have converged to the target weight values within a predetermined accuracy threshold value; and

update the array of weights stored in the array of cells of the resistive processing unit using the new set of programmed weight, in response to determining that the determined weight values have not converged to the target weight values within the predetermined accuracy threshold value.

10. A computer program product, comprising:

one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions comprising:

program instructions to store a given matrix of weight values in an array of cells of a resistive processing unit; and

program instructions to perform a weight extraction process which comprise:

program instructions to apply a set of input vectors to the resistive processing unit to perform analog matrix-vector multiplication operations on the given matrix;

program instructions to obtain a set of output vectors resulting from the analog matrix-vector multiplication operations; and

program instructions to determine weight values of the given matrix stored in the array of cells utilizing the set of input vectors and the set of output vectors, the determining comprising:

generating a first matrix comprising the set of input vectors;

generating a second matrix comprising the set of output vectors; and

multiplying the first matrix by a transpose of the first matrix to thereby generate a third matrix.

11. The computer program product of claim 10 , wherein the program instructions to apply the set of input vectors to the resistive processing unit to perform the analog matrix-vector multiplication operations comprise program instructions to apply a set of linearly independent input vectors to the resistive processing unit.

12. The computer program product of claim 10 , further comprising:

program instructions to perform a model training process which comprises applying a set of training vectors to the resistive processing unit to perform analog matrix-vector multiplication operations on the given matrix during the model training process to thereby generate a trained model which comprises the stored weight values of the given matrix in the array of cells; and

wherein the program instructions to perform the weight extraction process comprise program instructions to determine the weight values of the given matrix stored in the array of cells utilizing the set of training vectors and a set of resulting vectors that are generated as a result of the analog matrix-vector multiplication operations performed during the model training process.

13. The computer program product of claim 10 , wherein the program instructions to determine the weight values of the given matrix stored in the array of cells comprises program instructions to perform a multivariate linear regression computation using the set of input vectors and the set of output vectors to determine the weight values of the given matrix stored in the array of cells.

14. The computer program product of claim 10 , wherein the program instructions to determine the weight values of the given matrix stored in the array of cells further comprise:

program instructions to determine an inverse of the third matrix; and

program instructions to multiply the inverse of the third matrix, the first matrix, and a transpose of the second matrix to thereby generate a fourth matrix;

wherein a transpose of the fourth matrix comprises the determined weight values of the given matrix stored in the array of cells.

15. The computer program product of claim 10 , further comprising:

program instructions to compare the determined weight values with corresponding target weight values of the given matrix stored in the array of cells to determine differences between the determined weight values and the corresponding target weight values; and

program instruction to quantify one of a non-linear response and write noise associated with hardware elements of the resistive processing unit, based on the determined differences between the determined weight values and the corresponding target weight values;

program instructions to perform a second weight extraction process to determine second weight values of the given matrix stored in the array of cells;

program instructions to compare the determined weight values of the given matrix stored in the array of cells with corresponding ones of the second determined weight values of the given matrix stored in the array of cells to determine differences between the determined weight values and the corresponding second determined weight values; and

program instructions to quantify one of a non-linear response and write noise associated with the hardware elements of the resistive processing unit, based on the determined differences between the determined weight values and the corresponding second determined weight values.

16. The computer program product of claim 15 , further comprising:

program instructions to store the array of weights in the array of cells of the resistive processing unit using programmed weight values which correspond to the target weight values;

program instructions to estimate a transfer function which represents a relationship between the programmed weight values and the corresponding determined weight values determined as a result of performing the weight extraction process;

program instruction to apply the target weight values to an inverse of the transfer function to thereby determine a new set of programmed weights;

program instructions to determine whether the determined weight values have converged to the target weight values within a predetermined accuracy threshold value; and

program instructions to update the array of weights stored in the array of cells of the resistive processing unit using the new set of programmed weight, in response to determining that the determined weight values have not converged to the target weight values within the predetermined accuracy threshold value.

17. A method comprising:

storing a given matrix of weight values in an array of cells of a resistive processing unit; and

performing a weight extraction process which comprises:

applying a set of input vectors to the resistive processing unit to perform analog matrix-vector multiplication operations on the given matrix;

obtaining a set of output vectors resulting from the analog matrix-vector multiplication operations; and

determining weight values of the given matrix stored in the array of cells utilizing the set of input vectors and the set of output vectors, the determining comprising:

generating a first matrix comprising the set of input vectors;

generating a second matrix comprising the set of output vectors; and

multiplying the first matrix by a transpose of the first matrix to thereby generate a third matrix.

18. The method of claim 17 , wherein applying the set of input vectors to the resistive processing unit to perform the analog matrix-vector multiplication operations comprises applying a set of linearly independent input vectors to the resistive processing unit.

19. The method of claim 17 , wherein determining the weight values of the given matrix stored in the array of cells, further comprises:

determining an inverse of the third matrix; and

multiplying the inverse of the third matrix, the first matrix, and a transpose of the second matrix to thereby generate a fourth matrix;

wherein a transpose of the fourth matrix comprises the determined weight values of the given matrix stored in the array of cells.

20. The method of claim 17 , further comprising:

storing the array of weights in the array of cells of the resistive processing unit using programmed weight values which correspond to target weight values;

estimating a transfer function which represents a relationship between the programmed weight values and the corresponding determined weight values determined as a result of performing the weight extraction process;

applying the target weight values to an inverse of the transfer function to thereby determine a new set of programmed weights;

determining whether the determined weight values have converged to the target weight values within a predetermined accuracy threshold value; and

updating the array of weights stored in the array of cells of the resistive processing unit using the new set of programmed weight, in response to determining that the determined weight values have not converged to the target weight values within the predetermined accuracy threshold value.

Assignments (1)

Continuity (1)

References Cited (18)