Apparatus including multiple high bandwidth memory cubes
An apparatus including a high bandwidth memory circuit and associated systems and methods are disclosed herein. The apparatus may include multiple HBM cubes connected to a processor, such as a GPU. The HBM cubes may be connected in series or in parallel. One or more of the HBM cubes can include a secondary communication circuit configured to facilitate the expanded connection between the multiple cubes.
1 . An apparatus, comprising:
an interposer having at least a first input and output (IO) circuit and a second IO circuit;
a processing unit mounted on the interposer and connected to the first IO circuit;
a first memory cube including first local memory, second local memory, and at least a first physical layer circuit and a secondary communication circuit, the first memory cube mounted on the interposer with the first physical layer circuit connected to the processing unit by the first IO circuit;
a second memory cube mounted on the interposer and connected to the secondary communication circuit of the first memory cube by the second IO circuit, wherein:
the first memory cube is configured to:
receive, from the processing unit, a command through the first IO circuit and the first physical layer circuit;
identify that an address of a storage location identified by the command is in the second memory cube;
buffer the command at the secondary communication circuit; and
transfer the command to the second memory cube through the second IO circuit and the secondary communication circuit.
2 . The apparatus of claim 1 , wherein the second memory cube is further configured to:
receive, from the first memory cube, the command through the second IO circuit;
identify that the address of the storage location identified by the command is in the second memory cube; and
access the storage location to perform a read operation or a write operation.
3 . The apparatus of claim 1 , wherein:
the command is a first command,
the first memory cube includes a second physical layer circuit,
the second physical layer circuit is connected to the processing unit by a third IO circuit,
the second physical layer circuit is connected to the second memory cube by a fourth IO circuit, and
the first memory cube is further configured to:
receive, from the processing unit, a second command through the third IO circuit and the second physical layer circuit;
identify that an address of a storage location identified by the second command is in the second memory cube; and
transfer the second command to the second memory cube through the fourth IO circuit and the second physical layer circuit.
4 . The apparatus of claim 1 , wherein the first memory cube includes:
a set of core dies configured to locally store data; and
an interface die coupled to the set of core dies, the interface die including:
the first physical layer circuit configured to provide a communication interface to the processing unit, and
the secondary communication circuit configured to provide a communication interface to the second memory cube including its own local memory, wherein the secondary communication circuit is configured to (1) implement a serial connection with the second memory cube and the processing unit or (2) complete a connection between the processing unit and the second memory cube that is parallel to a connection between the first physical layer circuit and the processing unit.
5 . The apparatus of claim 1 , wherein the first memory cube and the second memory cube are connected in series relative to the processing unit.
6 . The apparatus of claim 1 , wherein the first memory cube and the second memory cube are connected in parallel relative to the processing unit.
7 . The apparatus of claim 1 , wherein the first physical layer circuit and the secondary communication circuit are connected with one or more through silicon vias.
8 . The apparatus of claim 1 , wherein:
the first local memory corresponds to a first rank of memory locations accessible to the processing unit; and
the second local memory corresponds to a second rank of memory locations accessible to the processing unit.
9 . A memory device, comprising:
a physical layer circuit configured to communicate with a first device external to the memory device;
a secondary communication circuit configured to communicate with at least a second device that is external to the memory device;
a logic that is coupled to the physical layer circuit and the secondary communication circuit,
wherein the logic is configured to:
receive, from the first device, a command through the physical layer circuit, wherein the command is associated with a storage location;
identify that an address of the storage location is in the second device; and
transfer the command to the second device through the secondary communication circuit.
10 . The memory device of claim 9 , wherein the command is a first command, and wherein the memory device is further configured to:
receive, from the first device, a second command through the secondary communication circuit;
identify that an address of a storage location identified by the second command is in the second device; and
transfer the second command to the second device through the secondary communication circuit.
11 . The memory device of claim 9 , wherein the memory device further comprises:
a set of core dies configured to locally store data; and
an interface die coupled to the set of core dies, the interface die including:
the physical layer circuit configured to provide a communication interface to the first device, and
the secondary communication circuit is configured to provide a communication interface to the second device including its own local memory, wherein the secondary communication circuit is configured to (1) implement a serial connection with the second device and the first device or (2) complete a connection between the first device and the second device that is parallel to a connection between the physical layer circuit and the first device.
12 . The memory device of claim 11 , wherein:
the set of core dies include memory cells that are organized into one or more ranks; and
identify that the address corresponds to a rank that is different than the one or more ranks corresponding to the memory cells.
13 . The memory device of claim 9 , wherein the secondary communication circuit includes a buffer configured to temporarily buffer the command before transferring the command to the second device.
14 . A method of operating an apparatus, the method comprising:
receiving, by first physical layer circuit on a first memory cube, a command that is from a processing unit and communicated through a first input and output (IO) circuit on an interposer, wherein the first memory cube and the processing unit are mounted on the interposer;
identifying, at the first memory cube, that an address of a storage location identified by the command is in a second memory cube;
buffering the command at a secondary communication circuit within the first memory cube; and
transferring the command to the second memory cube through a second IO circuit on the interposer and the secondary communication circuit.
15 . The method of claim 14 , wherein:
the command is a first command,
the first memory cube includes a second physical layer circuit,
the second physical layer circuit is connected to the processing unit by a third IO circuit,
the second physical layer circuit is connected to the second memory cube by a fourth IO circuit, and
the method further comprising:
receiving, from the processing unit, a second command through the third IO circuit and the second physical layer circuit;
identifying that an address of a storage location identified by the second command is in the second memory cube; and
transferring the second command to the second memory cube through the fourth IO circuit and the second physical layer circuit.
16 . The method of claim 14 , wherein the command is a first command, the method further comprising:
receiving a second command from the processing unit and the first IO circuit; and
determining that the second command is associated with a memory location within a set of core dies within the first memory cube and configured to locally store data.
17 . The method of claim 14 , wherein:
receiving the command includes operating the first physical layer circuit on an interface die within the first memory cube to provide a communication interface to the processing unit, and
buffering and transferring the command includes operating the secondary communication circuit to provide a communication interface to the second memory cube based on (1) implementing a serial connection with the second memory cube and the processing unit or (2) completing a connection between the processing unit and the second memory cube that is parallel to a connection between the first physical layer circuit and the processing unit.
18 . The method of claim 14 , wherein the first memory cube and the second memory cube are serially connected to the processing unit.
19 . The method of claim 14 , wherein the first memory cube and the second memory cube are parallelly connected to the processing unit.
20 . The method of claim 14 , wherein the first physical layer circuit and the secondary communication circuit are connected with one or more through silicon vias.