Granted Patent B2

US 12,334,183 · App. 18/306,531 · Granted Jun 17, 2025

Inventors: Kevin Skadron (Charlottesville, VA); Marzieh Lenjani (Champaign, IL)

Assignee: UNIVERSITY OF VIRGINIA PATENT FOUNDATION

G11C7/1039G06F17/16G11C7/1057

Abstract

Claims (23)

1. An integrated circuit memory device comprising:

a plurality of banks of memory, each of the banks of memory including:

a first pair of sub-arrays comprising first and second sub-arrays, the first pair of sub-arrays configured to store data in memory cells of the first pair of sub-arrays;

a first row buffer memory circuit located in the integrated circuit memory device adjacent to the first pair of sub-arrays and configured to store first row data received from the first pair of sub-arrays and configured to transfer the first row data into and/or out of the first row buffer memory circuit; and

a first sub-array level processor circuit in the integrated circuit memory device adjacent to the first pair of sub-arrays and operatively coupled to the first row data, wherein the first sub-array level processor circuit is configured to perform column oriented processing a sparse matrix kernel stored, at least in-part, in the first pair of sub-arrays, with input vector values stored, at least in part, in the first pair of sub-arrays to provide output vector values representing products of values stored in columns of the sparse matrix kernel with the input vector values.

2. The integrated circuit memory device of claim 1 wherein the values in long columns of the sparse matrix kernel and the output vector values representing accumulated values of the values in the long columns activated by the input vector values, are partitioned within the memory device so that both are stored in the same sub-array of the memory.

3. The integrated circuit memory device of claim 2 wherein short columns of the sparse matrix kernel are not partitioned within the memory device so that both are stored in the same sub-array of the memory.

4. The integrated circuit memory device of claim 2 wherein accumulation operations for the values in the long columns activated by the input vector values partitioned within the same sub-array comprise local accumulations and wherein the accumulation operations for the values in the long columns activated by the input vector values not partitioned within the same sub-array comprise local accumulations comprise remote accumulations.

5. The integrated circuit memory device of claim 4 further comprising:

a pair of dispatch sub-arrays configured to store remote accumulations to be dispatched to a remote level processor circuit including a remote sub-array level processor circuit for accumulation;

a dispatch sub-array level processor circuit operatively coupled to the pair of dispatch sub-arrays and configured to dispatch the remote accumulations from the pair of dispatch sub-arrays to the remote sub-array level processor circuit for accumulation responsive to an indication that the remote sub-array level processor circuit is idle.

6. The integrated circuit memory device of claim 5 further comprising:

indexing registers configured to represent an index value for the sparse matrix kernel beyond which accumulation operations are determined to be remote accumulations.

7. The integrated circuit memory device of claim 6 wherein the plurality of banks comprise a plurality of layers in a 3D stack.

8. The integrated circuit memory device of claim 7 wherein at least one layer in the plurality of layers comprises a logic layer configured to store the remote accumulations.

9. The integrated circuit memory device of claim 2 wherein the input vector values that activated the values in the long columns of the sparse matrix kernel are stored in the first sub-array of the memory.

10. The integrated circuit memory device of claim 7 further comprising:

a through-via extending through the plurality of layers to electrically couple data to/from the plurality of banks located among the plurality of layers.

11. The integrated circuit memory device of claim 1 wherein the first sub-array level processor circuit includes:

a control circuit operatively coupled to the first pair of sub-arrays, to the first row buffer memory circuit, and to the first sub-array level processor circuit and configured to select inputs to first and second inputs to the first sub-array level processor circuit and to provide instruction an ALU circuit included in the control circuit to operate on data in the first row buffer memory circuit.

12. The integrated circuit memory device of claim 1 wherein the first sub-array level processor circuit is configured to shift row data for the first pair of sub-arrays from or to the first row buffer memory circuit using a hot-one encoded value to randomly access a column of data stored in the first row buffer memory circuit.

13. An integrated circuit memory device comprising:

a sub-array level processor circuit in the integrated circuit memory device located adjacent to a pair of sub-arrays of the memory device and configured to perform column oriented processing on a sparse matrix kernel stored, at least in-part, in the pair of sub-arrays, with input vector values stored, at least in part, in the pair of sub-arrays to provide output vector values representing products of values stored in columns of the sparse matrix kernel with the input vector values.

Assignments (3)

Continuity (2)

References Cited (21)