Computational storage device for deep-learning recommendation system and method of operating the same
A computational storage device includes a nonvolatile memory configured to store a plurality of embedding tables for a deep-learning recommendation system (DLRS), and a storage controller configured to control an operation of the nonvolatile memory, store a plurality of applications that are off-loaded from a host device executing the DLRS, and support an execution of the DLRS by executing the plurality of applications and performing a plurality of calculations based on the plurality of embedding tables. The storage controller includes a machine learning engine configured to determine a management scheme of at least one embedding table of the plurality of embedding tables and the plurality of applications by analyzing the at least one embedding table and the plurality of applications.
1 . A computational storage device comprising:
a nonvolatile memory configured to store a plurality of embedding tables for a deep-learning recommendation system (DLRS); and
a storage controller configured to:
control an operation of the nonvolatile memory;
store a plurality of applications that are off-loaded from a host device executing the DLRS; and
support an execution of the DLRS by executing the plurality of applications and performing a plurality of calculations based on the plurality of embedding tables,
wherein the storage controller includes a machine learning engine configured to determine a management scheme of at least one embedding table of the plurality of embedding tables and the plurality of applications by analyzing the at least one embedding table and the plurality of applications, and
wherein the machine learning engine is further configured to perform an analysis of usage of the at least one embedding table by determining a number of times the at least one embedding table is used.
2 . The computational storage device of claim 1 , wherein the machine learning engine is further configured to:
classify the at least one embedding table as a first embedding table or a second embedding table based on the analysis and a reference usage number.
3 . The computational storage device of claim 2 , wherein the nonvolatile memory includes:
a first block including single-level memory cells (SLCs), each SLC storing one data bit; and
a second block including multi-level memory cells (MLCs), each MLC storing two or more data bits, and
wherein first embedding tables are stored in the first block and second embedding tables are stored in the second block.
4 . The computational storage device of claim 1 , further comprising a buffer memory configured to temporarily store data stored in or to be stored in the nonvolatile memory,
wherein the machine learning engine is configured to:
analyze usage of a plurality of vector data included in the plurality of embedding tables;
detect first vector data from among the plurality of vector data based on a result of analyzing the usage of the plurality of vector data; and
temporarily store the first vector data in the buffer memory, and
wherein the first vector data is used by loading from the buffer memory.
5 . The computational storage device of claim 1 , further comprising a reconfigurable hardware configured to store and execute at least one of the plurality of applications,
wherein the machine learning engine is configured to:
analyze usage of a plurality of operators associated with the plurality of applications;
detect a first application from among the plurality of applications; and
store the first application in the reconfigurable hardware, and
wherein the first application is used by the reconfigurable hardware.
6 . The computational storage device of claim 5 , wherein the reconfigurable hardware is disposed inside the storage controller.
7 . The computational storage device of claim 5 , wherein the reconfigurable hardware is disposed outside the storage controller.
8 . The computational storage device of claim 1 , wherein the machine learning engine includes:
a mode configuration module configured to receive operation mode information;
a retrieving module configured to receive a plurality of internal information from the nonvolatile memory;
a feature analyzing module configured to perform an analyzing operation on the plurality of internal information based on the plurality of embedding tables;
a clustering module configured to classify at least some of the plurality of internal information into a first cluster and a second cluster by performing a clustering operation based on a result of the analyzing an operation on the plurality of internal information;
a categorizing module configured to:
receive a plurality of pattern information,
select a first reference value and a second reference value from the first cluster and the second cluster, respectively, based on a result of the clustering operation and the plurality of pattern information, and
categorize the at least some of the plurality of internal information into a first reference group and a second reference group based on the first reference value and the second reference value; and
an estimating module configured to perform an estimating operation for analyzing the plurality of pattern information using the first reference group and the second reference group,
wherein the clustering module is configured to select an optimal clustering value by calculating a lowest similarity value between clusters.
9 . The computational storage device of claim 8 , wherein:
the clustering module is further configured to classify remaining information of the plurality of internal information that is not classified into the first and second clusters into a third cluster;
the categorizing module is further configured to select a third reference value from the third cluster and categorize the remaining information into a third reference group based on the third reference value; and
the estimating module is further configured to perform the estimating operation using the first, second, and third reference groups.
10 . The computational storage device of claim 8 , wherein the clustering module is further configured to:
classify the at least some of the plurality of internal information into the first cluster and the second cluster by performing a k-means clustering operation; and
determine the optimal clustering value representing an optimal quantity of clusters using an elbow technique.
11 . The computational storage device of claim 10 , wherein the categorizing module is configured to:
select internal information having a largest k-mean among internal information included in the first cluster as the first reference value;
select internal information having a largest k-mean among internal information included in the second cluster as the second reference value; and
categorize the at least some of the plurality of internal information into the first reference group and the second reference group by performing a support vector machine (SVM) operation on the internal information included in the first cluster and the internal information included in the second cluster based on the first reference value and the second reference value.
12 . The computational storage device of claim 8 , wherein the plurality of internal information include workload information associated with features of the plurality of applications, input/output (I/O) pattern information associated with features of accessing the computational storage device by the host device, and nonvolatile memory information associated with features of the nonvolatile memory.
13 . The computational storage device of claim 8 , wherein the feature analyzing module includes:
a first feature analyzer configured to perform a first feature analyzing operation without the plurality of embedding tables;
a second feature analyzer configured to perform a second feature analyzing operation using the plurality of embedding tables; and
a third feature analyzer configured to perform a third feature analyzing operation based on a result of the first feature analyzing operation and a result of the second feature analyzing operation.
14 . The computational storage device of claim 8 , wherein the machine learning engine further includes a reporting module configured to update the plurality of internal information based on a result of the estimating operation.
15 . The computational storage device of claim 1 , wherein the storage controller further includes:
a host interface configured to communicate with the host device; and
a program slot configured to store the plurality of applications.
16 . The computational storage device of claim 15 , wherein the host interface is configured to operate based on a nonvolatile memory express (NVMe) technical proposal (TP) 4091 protocol.
17 . The computational storage device of claim 15 , further comprising a buffer memory configured to temporarily store data stored in or to be stored in the nonvolatile memory, and
wherein the storage controller further includes a buffer manager configured to manage the buffer memory.
18 . The computational storage device of claim 17 , wherein the host interface is configured to operate based on compute express link (CXL) protocol.
19 . A method of operating a computational storage device, the method comprising:
storing a plurality of applications that are off-loaded from a host device executing a deep-learning recommendation system (DLRS);
storing a plurality of embedding tables that are used in the DLRS;
supporting an execution of the DLRS by executing the plurality of applications and performing a plurality of calculations based on the plurality of embedding tables; and
determining, by use of a machine learning engine, a management scheme of at least one of the plurality of embedding tables and the plurality of applications by analyzing the at least one of the plurality of embedding tables and the plurality of applications,
wherein the machine learning engine is configured to perform an analysis of usage of the at least one embedding table by determining a number of times the at least one embedding table is used.
20 . A computational storage device comprising:
a nonvolatile memory configured to store a plurality of embedding tables for a deep-learning recommendation system (DLRS);
a buffer memory configured to temporarily store data stored in or to be stored in the nonvolatile memory;
a storage controller configured to:
control an operation of the nonvolatile memory,
store a plurality of applications that are off-loaded from a host device executing the DLRS, and
support an execution of the DLRS by executing the plurality of applications and performing a plurality of calculations based on the plurality of embedding tables; and
a reconfigurable hardware configured to store and execute at least one of the plurality of applications, the reconfigurable hardware being disposed inside or outside the storage controller,
wherein the storage controller includes a machine learning engine configured to:
analyze at least one of the plurality of embedding tables and the plurality of applications by performing a k-means clustering operation, a support vector machine (SVM) operation, and an estimating operation based on a plurality of internal information; and
determine a management scheme of the at least one of the plurality of embedding tables and the plurality of applications based on a result of analyzing the at least one of the plurality of embedding tables and the plurality of applications, and
wherein the machine learning engine is further configured to:
analyze usage of the plurality of embedding tables and classify the plurality of embedding tables into first embedding tables stored in single-level memory cells (SLCs) and second embedding tables stored in multi-level memory cells (MLCs), or
analyze usage of a plurality of vector data included in the plurality of embedding tables, temporarily store at least one of the plurality of vector data in the buffer memory, and use the at least one of the plurality of vector data stored in the buffer memory, or
analyze usage of a plurality of operators associated with the plurality of applications, store at least one of the plurality of applications in the reconfigurable hardware, and use the at least one of the plurality of applications stored in the reconfigurable hardware.