Compute optimization mechanism
An apparatus to facilitate compute optimization is disclosed. The apparatus includes a mixed precision core including mixed-precision execution circuitry to execute one or more of the mixed-precision instructions to perform a mixed-precision dot-product operation comprising to perform a set of multiply and accumulate operations.
1. An apparatus comprising:
an interconnect fabric comprising one or more fabric switches;
a memory controller coupled to the interconnect fabric;
an input/output (IO) interface coupled to the interconnect fabric;
an array of multiprocessors coupled to the interconnect fabric to process mixed-precision instructions, at least one multiprocessor comprising:
a plurality of packed data registers to store packed floating-point source values at a first precision using a first number of bits, and to store at least one packed floating-point value at a second precision using a second number of bits equal to at least twice the first number of bits; and
mixed-precision execution circuitry to execute one or more of the mixed-precision instructions to perform a mixed-precision dot-product operation comprising to perform a set of multiply and accumulate operations including an operation D=A*B+C, wherein A, B, C, and D are matrix elements, A and B are floating-point values at the first precision, C is a floating-point value at the second precision, A is identified by a first source operand, B is identified by a second source operand, and C is identified by a third source operand.
2. The apparatus of claim 1 , further comprising:
a semiconductor substrate;
a parallel processor die comprising the interconnect fabric, memory controller, input/output (IO) interface, and array of multiprocessors mounted on the semiconductor substrate;
a 3D memory stack comprising a plurality of stacked memory dies mounted on the semiconductor substrate; and
a local memory interconnect to couple the memory controller to the 3D memory stack, the local memory interconnect comprising independent groups of memory interfaces, the independent groups of memory interfaces associated with respective memory dies of the plurality of stacked memory dies.
3. The apparatus of claim 2 , wherein a memory interface comprises a memory channel and wherein an independent group of memory interfaces comprises at least one physical memory channel and one or more virtual memory channels between a corresponding memory die and a multiprocessor.
4. The apparatus of claim 2 , further comprising:
a cache hierarchy to store data for the array of multiprocessors, the cache hierarchy including an L1 cache and an L2 cache to be shared between the array of multiprocessors.
5. The apparatus of claim 4 , further comprising:
memory management circuitry to allocate physical memory of the 3D memory stack as system memory.
6. The apparatus of claim 5 , further comprising:
an input/output memory management unit (IOMMU) coupled to the interconnect fabric, the IOMMU comprising a translation buffer to store virtual-to-physical address translations to access the system memory, including the 3D memory stack.
7. The apparatus of claim 6 , wherein a first one or more virtual-to-physical address translations are to identify regions in the 3D memory stack and wherein a second one or more virtual-to-physical address translations are to identify regions in a system memory device.
8. The apparatus of claim 7 , wherein the array of multiprocessors is to connect to the system memory device via the IO interface.
9. The apparatus of claim 1 , wherein the mixed-precision instructions are primitives of a machine learning framework.
10. The apparatus of claim 9 , wherein the matrix elements are elements of matrices associated with a convolutional layer.
11. The apparatus of claim 10 , wherein the matrices associated with the convolutional layer comprise a first matrix and a second matrix and the set of multiply and accumulate operations include a multiplication of a packed data element from the first matrix and a packed data element from the second matrix.
12. The apparatus of claim 1 , further comprising:
virtualization circuitry to share the array of multiprocessors with a plurality of virtual machines.
13. The apparatus of claim 12 , wherein the virtualization circuitry comprises multiple sets of control registers to be associated with multiple corresponding virtual machines, a group of control registers to store one or more address pointers to identify a region of memory associated with a corresponding virtual machine.
14. The apparatus of claim 1 , wherein the mixed-precision execution circuitry is to generate an output matrix.
15. The apparatus of claim 14 , wherein the mixed-precision execution circuitry is to evaluate an activation function based on the output matrix.
16. A system comprising:
a semiconductor substrate;
a 3D memory stack comprising a plurality of stacked memory dies mounted on the semiconductor substrate;
a local memory interconnect to couple a memory controller to the 3D memory stack, the local memory interconnect comprising independent groups of memory interfaces, the independent groups of memory interfaces associated with respective memory dies of the plurality of stacked memory dies;
a parallel processor die comprising an interconnect fabric, the memory controller, an input/output (IO) interface, and array of multiprocessors mounted on the semiconductor substrate, wherein the array of multiprocessors are configured to process mixed-precision instructions, at least one multiprocessor comprising:
a plurality of packed data registers to store packed floating-point source values at a first precision using a first number of bits, and to store at least one packed floating-point value at a second precision using a second number of bits equal to at least twice the first number of bits; and
mixed-precision execution circuitry to execute one or more of the mixed-precision instructions to perform a mixed-precision dot-product operation comprising to perform a set of multiply and accumulate operations including an operation D=A*B+C, wherein A, B, C, and D are matrix elements, A and B are floating-point values at the first precision, and C is a floating-point value at the second precision, A is identified by a first source operand, B is identified by a second source operand, and C is identified by a third source operand.
17. The system of claim 16 , wherein a memory interface comprises a memory channel and wherein an independent group of memory interfaces comprises at least one physical memory channel and one or more virtual memory channels between a corresponding memory die and a multiprocessor.
18. The system of claim 16 , further comprising:
a cache hierarchy to store data for the array of multiprocessors, the cache hierarchy including an L1 cache and an L2 cache to be shared between the array of multiprocessors.
19. The system of claim 18 , further comprising:
memory management circuitry to allocate physical memory of the 3D memory stack as system memory.
20. The system of claim 19 , further comprising:
an input/output memory management unit (IOMMU) coupled to the interconnect fabric, the IOMMU comprising a translation buffer to store virtual-to-physical address translations to access the system memory, including the 3D memory stack, wherein a first one or more virtual-to-physical address translations are to identify regions in the 3D memory stack, a second one or more virtual-to-physical address translations are to identify regions in a system memory device, and the array of multiprocessors are to connect to the system memory device via the IO interface.