IP Library › Granted Patent US 12675434
Granted Patent B2
US 12675434 · App. 18/542,985 · Granted Jul 7, 2026

Irregular cadence data processing units

Inventors: Indranil Chakraborty (Mountain View, CA); Rahul Nagarajan (San Jose, CA); Christopher Aaron Clark (Madison, WI)
Assignee: Google LLC
G06F15/8053G06F15/8061G06F17/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12675434
App. No.
18/542,985
Granted
Jul 7, 2026
Kind
B2
Abstract

Aspects of the disclosure are directed to an architecture including a dynamic serialization buffer and/or dynamic deserialization buffer coupled between a vector processing unit and a matrix multiplication unit. The dynamic serialization buffer and/or dynamic deserialization buffer allow for streaming any integer of vectors per cycle when performing acceleration of matrix multiplication operations. The matrix multiplication unit receives vectors equivalent to an amount of data from the vector processing unit at an arbitrary rate of vectors per cycle. The matrix multiplication unit processes the vectors to generate resulting vectors that are output at the arbitrary rate.

Claims (48)

1 . An architecture for arbitrary throughput of matrix multiplication, the architecture comprising:

a dynamic serialization buffer coupled to an input of a matrix multiplication unit and a dynamic deserialization buffer coupled to an output of the matrix multiplication unit;

the dynamic serialization buffer configured to:

receive a first amount of data from a first vector register;

output a second amount of data, the second amount of data being less than the first amount of data, at each of a number of cycles until the dynamic serialization buffer contains an amount of data less than the second amount of data; and

based on containing the amount of data less than the second amount of data, receive a subsequent first amount of data from the first vector register or a second vector register;

the matrix multiplication unit configured to:

receive the second amount of data;

process the second amount of data to generate a third amount of data representing results of the matrix multiplication; and

output the third amount of data; and

the dynamic deserialization buffer configured to:

receive the third amount of data at each of the number of cycles until the dynamic deserialization buffer contains an amount of data greater than a fourth amount of data, the fourth amount of data being greater than the third amount of data; and

based on containing the amount of data greater than the fourth amount of data, output the fourth amount of data to the first vector register, the second vector register, or a third vector register.

2 . The architecture of claim 1 , further comprising a vector processing unit comprising at least one of the first vector register, second vector register, or third vector register.

3 . The architecture of claim 1 , wherein the matrix multiplication unit is a systolic array.

4 . The architecture of claim 1 , wherein the first amount of data is equal to a lane dimension multiplied by a sublane dimension multiplied by a number of elements per vector register.

5 . The architecture of claim 1 , wherein the second amount of data is equal to an arbitrary amount of vectors per cycle.

6 . The architecture of claim 1 , wherein the third amount of data is equal to an arbitrary amount of vectors per cycle.

7 . The architecture of claim 1 , wherein the second amount of data and the third amount of data are equivalent.

8 . The architecture of claim 1 , wherein the fourth amount of data is equal to a lane dimension multiplied by a sublane dimension multiplied by a number of elements per vector register.

9 . The architecture of claim 1 , wherein the dynamic serialization buffer has a depth based on a maximum amount of data that the amount of data less than the second amount of data can be.

10 . The architecture of claim 1 , wherein the dynamic deserialization buffer has a depth based on a maximum amount of data that can remain in the deserialization buffer when the fourth amount of data is output.

11 . A method for operating an architecture for arbitrary throughput of matrix multiplication, the architecture including a dynamic serialization buffer coupled to an input of a matrix multiplication unit and a dynamic deserialization buffer coupled to an output of the matrix multiplication unit, the method comprising:

receiving, by the dynamic serialization buffer, a first amount of data from a first vector register;

outputting, by the dynamic serialization buffer, a second amount of data, the second amount of data being less than the first amount of data, at each of a number of cycles until the dynamic serialization buffer contains an amount of data less than the second amount of data;

based on containing the amount of data less than the second amount of data, receiving, by the dynamic serialization buffer, a subsequent first amount of data from the first vector register or a second vector register;

receiving, by the matrix multiplication unit, the second amount of data;

processing, by the matrix multiplication unit, the second amount of data to generate a third amount of data representing results of the matrix multiplication;

outputting, by the matrix multiplication unit, the third amount of data;

receiving, by the dynamic deserialization buffer, the third amount of data at each of the number of cycles until the dynamic deserialization buffer contains an amount of data greater than a fourth amount of data, the fourth amount of data being greater than the third amount of data; and

based on containing the amount of data greater than the fourth amount of data, outputting, by the dynamic deserialization buffer, the fourth amount of data to the first vector register, the second vector register, or a third vector register.

12 . The method of claim 11 , wherein the first amount of data is equal to a lane dimension multiplied by a sublane dimension multiplied by a number of elements per vector register.

13 . The method of claim 11 , wherein the second amount of data is equal to an arbitrary amount of vectors per cycle.

14 . The method of claim 11 , wherein the third amount of data is equal to an arbitrary amount of vectors per cycle.

15 . The method of claim 11 , wherein the fourth amount of data is equal to a lane dimension multiplied by a sublane dimension multiplied by a number of elements per vector register.

16 . A non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations for operating an architecture for arbitrary throughput of matrix multiplication, the architecture including a dynamic serialization buffer coupled to an input of a matrix multiplication unit and a dynamic deserialization buffer coupled to an output of the matrix multiplication unit, the operations comprising:

receiving, by the dynamic serialization buffer, a first amount of data from a first vector register;

outputting, by the dynamic serialization buffer, a second amount of data, the second amount of data being less than the first amount of data, at each of a number of cycles until the dynamic serialization buffer contains an amount of data less than the second amount of data;

based on containing the amount of data less than the second amount of data, receiving, by the dynamic serialization buffer, a subsequent first amount of data from the first vector register or a second vector register;

receiving, by the matrix multiplication unit, the second amount of data;

processing, by the matrix multiplication unit, the second amount of data to generate a third amount of data representing results of the matrix multiplication;

outputting, by the matrix multiplication unit, the third amount of data;

receiving, by the dynamic deserialization buffer, the third amount of data at each of the number of cycles until the dynamic deserialization buffer contains an amount of data greater than a fourth amount of data, the fourth amount of data being greater than the third amount of data; and

based on containing the amount of data greater than the fourth amount of data, outputting, by the dynamic deserialization buffer, the fourth amount of data to the first vector register, the second vector register, or a third vector register.

17 . The non-transitory computer readable medium of claim 16 , wherein the first amount of data is equal to a lane dimension multiplied by a sublane dimension multiplied by a number of elements per vector register.

18 . The non-transitory computer readable medium of claim 16 , wherein the second amount of data is equal to an arbitrary amount of vectors per cycle.

19 . The non-transitory computer readable medium of claim 16 , wherein the third amount of data is equal to an arbitrary amount of vectors per cycle.

20 . The non-transitory computer readable medium of claim 16 , wherein the fourth amount of data is equal to a lane dimension multiplied by a sublane dimension multiplied by a number of elements per vector register.