Vector processing unit
A vector processing unit is described, and includes processor units that each include multiple processing resources. The processor units are each configured to perform arithmetic operations associated with vectorized computations. The vector processing unit includes a vector memory in data communication with each of the processor units and their respective processing resources. The vector memory includes memory banks configured to store data used by each of the processor units to perform the arithmetic operations. The processor units and the vector memory are tightly coupled within an area of the vector processing unit such that data communications are exchanged at a high bandwidth based on the placement of respective processor units relative to one another, and based on the placement of the vector memory relative to each processor unit.
1. A system comprising:
a first vector processing unit (VPU) lane;
a vector memory co-located with the first VPU lane, the vector memory having a plurality of memory banks;
a second VPU lane that is within a distance to the vector memory co-located with the first VPU lane such that data traverses the distance in a single clock cycle; and
a matrix unit configured to perform matrix multiplication on data that is received from the first VPU lane and the second VPU lane.
2. The system of claim 1 , wherein each of the first VPU lane and the second VPU lane includes a respective vector memory having a plurality of memory banks.
3. The system of claim 1 , wherein each of the first VPU lane and the second VPU lane is a VPU sub-lane.
4. The system of claim 1 , wherein each of the first VPU lane and the second VPU lane is a respective computing resource of an integrated circuit die section of the system.
5. The system of claim 4 , wherein each of the first VPU lane and the second VPU lane comprises a plurality of VPU sub-lanes.
6. The system of claim 4 , wherein a first resource within a VPU sub-lane of the first VPU lane is within a distance to a second resource within the VPU sub-lane of the first VPU lane such that data traverses the distance in a single clock cycle.
7. The system of claim 6 , wherein a first resource within a VPU sub-lane of the second VPU lane is within a distance to a second resource within the VPU sub-lane of the second VPU lane such that data traverses the distance in a single clock cycle.
8. The system of claim 4 , further comprising:
an external memory coupled to each of the first VPU lane and the second VPU lane; and
an inter-chip interconnect that interconnects each of the external memory, the first VPU lane, and the second VPU lane.
9. The system of claim 8 , wherein the external memory is external to the integrated circuit die section.
10. The system of claim 8 , wherein each of the external memory and the inter-chip interconnect is configured to exchange data with the vector memory and the first VPU lane.
11. The system of claim 4 , wherein the vector memory is included in the first VPU lane.
12. The system of claim 11 , further comprising:
a plurality of second VPU lanes; and
a respective vector memory in each second VPU lane of the plurality of second VPU lanes.
13. The system of claim 4 , wherein:
the matrix unit is external to the integrated circuit die section; and
the data traverses a distance between the matrix unit and at least the first VPU lane in a single clock cycle.
14. The system of claim 4 , wherein the data comprises at least 1024 vector operands.
15. The system of claim 1 , wherein the data is represented as a multi-dimensional vector and the system further comprises:
a permute unit configured to reshape or rearrange the data with reference to the multi-dimensional vector.
16. The system of claim 1 , further comprising:
a cross-lane unit configured to move data between the first VPU lane and the second VPU lane.
17. A system comprising:
an external memory;
an inter-chip interconnect;
at least one vector processing unit (VPU) lane;
wherein each VPU lane of the at least one VPU lane comprises corresponding vector memory having a plurality of memory banks,
wherein each of the external memory and the inter-chip interconnect is configured to exchange data with the vector memory of the at least one VPU lane,
wherein each VPU lane of the at least one VPU lane comprises a plurality of VPU sub-lanes, and
wherein each VPU lane of the at least one VPU lane is within a distance to the corresponding vector memory such that data traverses the distance in a single clock cycle; and
a matrix unit configured to perform matrix multiplication on vector operands corresponding to the data, wherein the vector operands are received from the at least one VPU lane.
18. The system of claim 17 , wherein the data is represented as a multi-dimensional vector and the system further comprises:
a permute unit configured to reshape or rearrange the data with reference to the multi-dimensional vector, and
a cross-lane unit configured to move data between two or more VPU lanes of the system.