Memory expander, computing systems, and operating method of the host device
A memory expander is disclosed. The memory expander includes a memory, a memory controller configured to control the memory, a compute express link (CXL) engine configured to acquire a CXL flit from a host device connected to the memory expander and configured to acquire a calculation request for pieces of data stored in the memory by performing conversion on the CXL flit, and a domain-specific accelerator configured to perform a calculation in response to the calculation request.
1 . A memory expander comprising:
a memory;
a memory controller configured to control the memory;
a compute express link (CXL) engine configured to acquire a CXL flit associated with a calculation request for pieces of data stored in the memory by accessing a host memory of a host device connected to the memory expander and configured to acquire a command and data associated with the calculation request by performing conversion on the CXL flit; and
a domain-specific accelerator configured to perform a calculation between tensors corresponding to the calculation request based on the command and the data.
2 . The memory expander of claim 1 , wherein the memory expander is a type-3 device defined in a CXL protocol.
3 . The memory expander of claim 1 , further comprising:
an interface register configured to acquire a doorbell signal from the host device,
wherein the doorbell signal is a signal indicating that the command and the data associated with the calculation request are written to a host memory of the host device.
4 . The memory expander of claim 1 , wherein the domain-specific accelerator comprises:
a request queue configured to store the calculation request of the host device;
a scheduler configured to allocate the calculation request to at least one tensor calculation accelerator;
the at least one tensor calculation accelerator configured to perform a calculation between the tensors in response to the allocated calculation request;
a tensor reading module configured to read the tensors from the memory; and
a multiplexer configured to support the at least one tensor calculation accelerator to share the tensor reading module.
5 . The memory expander of claim 4 , wherein the at least one tensor calculation accelerator comprises:
at least one element-wise calculation module configured to perform a first operation on corresponding elements among elements forming each of the tensors; and
a tensor reduction module configured to perform a second operation on a result of the first operation.
6 . The memory expander of claim 4 , wherein
each of the tensors is divided and stored in a plurality of memory expanders comprising the memory expander, and
the at least one tensor calculation accelerator is configured to perform a calculation on at least a portion of the tensors.
7 . The memory expander of claim 4 , wherein
each of the tensors corresponds to an embedding vector, and
the at least one tensor calculation accelerator is configured to calculate a similarity between an input vector acquired from the host device and stored in the memory and a vector pre-stored in the memory.
8 . The memory expander of claim 7 , wherein the similarity corresponds to a Euclidean distance between the input vector and the vector or an angular distance between the input vector and the vector.
9 . A computing system comprising:
a host device configured to store a command and data associated with a calculation request in a host memory and configured to transmit, to a memory expander, a doorbell signal indicating that the command and the data associated with the calculation request are written to the host memory, and
the memory expander configured to perform a calculation between tensors corresponding to the calculation request in response to receiving the doorbell signal and configured to transmit a result of the calculation between the tensors to the host memory,
wherein the tensors are stored in the memory expander.
10 . The computing system of claim 9 , wherein the doorbell signal is received by an interface register of the memory expander.
11 . The computing system of claim 9 , wherein the memory expander is configured to:
acquire a compute express link (CXL) flit associated with the calculation request by accessing the host memory;
acquire the command and the data by performing conversion on the CXL flit; and
perform a calculation between the tensors based on the command and the data.
12 . The computing system of claim 11 , wherein the accessing of the memory expander to the host memory is direct memory access (DMA).
13 . The computing system of claim 11 , wherein the calculation between the tensors is performed by a domain-specific accelerator comprised in the memory expander.
14 . The computing system of claim 9 , wherein the memory expander is a type-3 device defined in a CXL protocol.
15 . An operating method of a host device configured to perform an approximate nearest neighbor search based on a search node, the operating method comprising:
acquiring graph data, wherein each of nodes of the graph data corresponds to tensors stored in a memory expander;
acquiring information of a neighbor node neighboring the search node by searching for the graph data;
acquiring a result of a calculation between the tensors stored in the memory expander and an input tensor input to the host device, based on the information of the neighbor node; and
updating a candidate array based on the result of the calculation,
wherein the candidate array comprises, among the tensors stored in the memory expander, information about a tensor that is similar to the input tensor.
16 . The operating method of claim 15 , wherein the calculation is performed by the memory expander, which is a type-3 device defined in a compute express link (CXL) protocol.
17 . The operating method of claim 15 , wherein the candidate array comprises the nodes comprised in the graph data,
wherein the nodes are mapping of a correspondence relationship between the nodes and the tensors stored in the memory expander, whether the nodes visit, and a calculation result between tensors corresponding to the nodes and the input tensor.
18 . The operating method of claim 15 , wherein the updating of the candidate array comprises:
inserting information into the candidate array;
sorting nodes of the candidate array into which the information is inserted; and
selecting a subsequent search node from the sorted nodes of the candidate array.
19 . The operating method of claim 18 , wherein the inserting of the information into the candidate array comprises:
displaying whether the search node visits;
adding the neighbor node to the candidate array; and
mapping the result of the calculation to the neighbor node.
20 . The operating method of claim 18 , wherein the selecting of the subsequent search node further comprises selecting, from among unvisited nodes in the sorted nodes of the candidate array, a node having a highest similarity with the input tensor as the subsequent search node.