Techniques for an efficient fabric attached memory
Fabric Attached Memory (FAM) provides a pool of memory that can be accessed by one or more processors, such as a graphics processing unit(s) (GPU)(s), over a network fabric. In one instance, a technique is disclosed for using imperfect processors as memory controllers to allow memory, which is local to the imperfect processors, to be accessed by other processors as fabric attached memory. In another instance, memory address compaction is used within the fabric elements to fully utilize the available memory space.
1. A fabric attached memory comprising:
a processor configured to communicate with an interconnect fabric; and
at least one memory operatively coupled to the processor,
the processor being structured to perform at least one read-modify-write atomic memory access command on the at least one memory,
wherein the processor is further configured such that a compute circuit capability thereof is programmatically or otherwise intentionally disabled.
2. The fabric attached memory of claim 1 wherein the compute circuit is fused to be intentionally disabled.
3. The fabric attached memory of claim 1 wherein the processor comprises at least one streaming multiprocessor.
4. The fabric attached memory of claim 1 wherein the interconnect fabric comprises NVIDIA NVLINK™.
5. The fabric attached memory of claim 1 wherein the at least one memory comprises an array of discrete semiconductor random access memory devices.
6. A fabric attached memory comprising:
a processor configured to communicate with an interconnect fabric; and
at least one memory operatively coupled to the processor,
the processor being structured to perform at least one read-modify-write atomic memory access command on the at least one memory,
wherein the processor is further configured such that a compute circuit capability thereof is defective, disabled or not present, and the processor includes a plurality of fabric interconnect ports only a subset of which are configured to be connected to the interconnect fabric.
7. A fabric attached memory comprising:
a processor configured to communicate with an interconnect fabric; and
at least one memory operatively coupled to the processor, the processor being structured to perform at least one read-modify-write atomic memory access command on the at least one memory, wherein the processor is further configured such that a compute circuit capability thereof is defective, disabled or not present, and the processor includes a boot ROM, a memory controller, a row remapper, a data cache, a crossbar connection and a fabric connection.
8. A fabric attached memory system comprising:
an interconnect fabric;
at least one source GPU interconnected to the interconnect fabric, the source GPU generating a memory address; and
plural fabric attached memories interconnected to the interconnect fabric, the plural fabric attached memories each defining an address space;
wherein the interconnection between the source GPU and the interconnect fabric and the interconnection between each the fabric attached memory and the interconnect fabric are asymmetrical; and
wherein at least one of the source GPU, the interconnect fabric and the plural fabric attached memories includes an address transformer that transforms the memory address the at least one source GPU generates into a fabric attached memory address space.
9. The fabric attached memory system of claim 8 wherein the address transformer comprises a division or compaction circuit.
10. The fabric attached memory system of claim 8 wherein the address transformer includes a swizzler that matches the swizzle performed by the source GPU, and an address compactor.
11. The fabric attached memory system of claim 10 wherein the at least one GPU swizzles the generated address in order to select an interconnect link within the interconnect fabric.
12. The fabric attached memory system of claim 8 wherein each fabric attached memory address space is less than an address space defined by the memory address the GPU generates.
13. A fabric attached memory baseboard comprising:
a printed circuit board;
a plurality of fabric attached memory modules disposed on the printed circuit board, each of the plurality of fabric attached memory modules connected to an interconnect fabric, and
a processor disposed on the printed circuit board, the processor managing the plurality of fabric attached memory modules;
wherein the plurality of fabric attached memory modules each are capable of performing GPU atomic memory operations and peer-to-peer GPU communications via the interconnect fabric while disaggregating the quantity of compute-capable GPUs from memory capacity provided by the fabric attached memory modules, wherein the plurality of fabric attached memory modules each include a floor swept GPU that is at least in part defective and/or fused to disable GPU compute operations.
14. A fabric attached memory baseboard comprising:
a printed circuit board;
a plurality of fabric attached memory modules disposed on the printed circuit board, each of the plurality of fabric attached memory modules connected to an interconnect fabric, and
a processor disposed on the printed circuit board, the processor managing the plurality of fabric attached memory modules;
wherein the plurality of fabric attached memory modules each are capable of performing GPU atomic memory operations and peer-to-peer GPU communications via the interconnect fabric while disaggregating the quantity of compute-capable GPUs from memory capacity provided by the fabric attached memory modules,
wherein the plurality of fabric attached memory modules each comprise a memory controller that has no GPU compute capability but comprises at least:
a boot ROM;
a memory controller capable of hardware-accelerating atomic memory commands without emulation;
a row remapper;
a data cache;
a crossbar interconnection; and
a fabric interconnect interface capable of peer-to-peer communication over the interconnect fabric with GPUs.
15. A fabric attached memory system comprising:
an interconnect fabric;
a graphics processing unit connected to the interconnect fabric, the graphics processing unit configured to provide a compute capability;
a first memory connected to the graphics processing unit;
a processing circuit configured to communicate with the interconnect fabric, the processing circuit including a boot ROM, a memory controller, a row remapper, a data cache, a crossbar connection, and an interconnect fabric connection and being structured to perform at least one read-modify-write atomic memory access command but configured not to provide said compute capability; and
a second memory connected to the processing circuit,
wherein the graphics processing unit is capable of atomically accessing the second memory via the interconnect fabric and the processing circuit.
16. The system of claim 15 wherein the compute capability comprises one or more of the following:
(a) atomic addition operating on floating point values in global and shared memory;
(b) warp vote and ballot functions;
(c) Memory Fence Functions;
(d) Synchronization Functions;
(e) Surface functions;
(f) 3D grid of thread blocks;
(g) funnel shift;
(h) dynamic parallelism;
(i) half-precision floating-point operations:
(j) addition, subtraction, multiplication, comparison, warp shuffle functions, conversion; and
(k) tensor core.