Method and apparatus for supporting machine learning algorithms and data pattern matching in ethernet SSD
A data storage device includes a memory array for storing data; a host interface for providing an interface with a host computer running an application; a central control unit configured to receive a command in a submission queue from the application and initiate a search process in response to a search query command; a preprocessor configured to reformat data contained in the search query command and generate a reformatted data; and one or more data processing units configured to extract one or more features from the reformatted data and perform a data operation on the data stored in the memory array in response to the search query command and return matching data from the data stored in the memory array to the application via the host interface.
1. A data storage device comprising:
a memory for storing data;
a host interface compatible with non-volatile memory express over fabrics (NVMeoF) and capable of receiving a command and a first data associated with the command from an external application, the first data comprising image data; and
a processor extracting a feature from the image data, performing a data operation on the data stored in the memory based on the command, identifying matching data among the data stored in the memory based on the feature, and returning the matching data to the external application via the host interface.
2. The data storage device of claim 1 , further comprising a preprocessor generating a reformatted data by reformatting the first data received from host interface based on at least one of the command and a type of the first data, and extracting the feature from the reformatted data.
3. The data storage device of claim 2 , wherein the first data is an image data, and the reformatted data is in a red/green/blue (RGB) format.
4. The data storage device of claim 1 , wherein the data storage device is attachable to a fabric compatible with the NVMeoF.
5. The data storage device of claim 1 , wherein the processor generates a binary code corresponding to the feature.
6. The data storage device of claim 1 , wherein the processor includes a convolution engine (CE) that extracts the feature in a convolutional neural network (CNN) layer.
7. The data storage device of claim 6 , wherein the convolution engine computes a neural descriptor for the CNN layer and compresses the neural descriptor.
8. The data storage device of claim 1 , wherein the processor further extracts a stored feature for the data stored in the memory and compares the stored feature with the feature of the first data.
9. The data storage device of claim 8 , wherein the processor further calculates a Hamming distance for the data stored in the memory.
10. The data storage device of claim 9 , wherein the processor further selects the matching data from the data stored in the memory based on the Hamming distance.
11. A method comprising:
receiving, at a data storage device, a command and a first data associated with the command from an external application via a host interface between a host computer, the host interface being compatible with non-volatile memory express over fabrics (NVMeoF), and the first data comprising image data;
extracting, in the data storage device, a feature from the image data;
performing, in the data storage device, a data operation on data stored in a memory of the data storage device based on the command;
identifying matching data among the data stored in the memory based on the feature; and
returning the matching data to the external application via the host interface.
12. The method of claim 11 , further comprising:
generating a reformatted data by reformatting the first data received from host interface based on at least one of the command and a type of the first data; and
extracting the feature from the reformatted data.
13. The method of claim 12 , wherein the first data is an image data, and the reformatted data is in a red/green/blue (RGB) format.
14. The method of claim 11 , wherein the data storage device is attachable to a fabric compatible with the NVMeoF.
15. The method of claim 11 , further comprising generating a binary code corresponding to the feature.
16. The method of claim 11 , further comprising:
extracting a stored feature for the data stored in the memory; and
comparing the stored feature with the feature of the first data.
17. The method of claim 16 , further comprising:
calculating a Hamming distance for the data stored in the memory.
18. The method of claim 17 , further comprising:
selecting the matching data from the data stored in the memory based on the Hamming distance.
19. A method comprising:
receiving, at a data storage device, a command and a first data associated with the command from an external application via a host interface between a host computer, the host interface being compatible with non-volatile memory express over fabrics (NVMeoF);
extracting, in the data storage device, a feature from the first data;
performing, in the data storage device, a data operation on data stored in a memory of the data storage device based on the command;
identifying matching data among the data stored in the memory based on the feature; and
returning the matching data to the external application via the host interface,
wherein the feature is extracted using a convolution engine of the data storage device, and the convolution engine extracts the feature in a convolutional neural network (CNN) layer.
20. The method of claim 19 , wherein the convolution engine computes a neural descriptor for the CNN layer and compresses the neural descriptor.