System and method for executing native client code in a storage device
A system and method for executing user-provided code securely on a solid state drive (SSD) to perform data processing on the SSD. In one embodiment, a user uses a security-oriented cross-compiler to compile user-provided source code for a data processing task on a host computer containing, or otherwise connected to, an SSD. The resulting binary is combined with lists of input and output file identifiers and sent to the SSD. A central processing unit (CPU) on the SSD extracts the binary and the lists of file identifiers. The CPU obtains from the host file system the addresses of storage areas in the SSD containing the data in the input files, reads the input data, executes the binary using a container, and writes the results of the data processing task back to the SSD, in areas corresponding to the output file identifiers.
1. A storage device communicatively coupled to a host through a storage interface, the storage device being configured to store host data provided by the host through the storage interface, the storage device comprising:
storage media;
a first processing unit; and
a program memory storing instructions that, when executed by the first processing unit, cause the first processing unit to:
instantiate, within the storage device, a device data processing agent and a container, wherein the device data processing agent is connected to the host through a virtual Transmission Control Protocol/Internet Protocol (TCP/IP) tunnel over the storage interface, and wherein the device data processing agent is configured to receive, from the host through the virtual TCP/IP tunnel over the storage interface, a first manifest comprising a first binary comprising first instructions;
extract the first binary, from the first manifest, within the storage device; and
execute the first binary to perform data processing on data stored in the storage device based on the first instructions in the first binary.
2. The storage device of claim 1 , wherein the container is configured to:
validate a safety of the first instructions, and
execute the first instructions when the first instructions are validated as safe.
3. The storage device of claim 1 , wherein the device data processing agent is further configured to receive data from the container and to store the received data in a storage area identified by a block address of a plurality of block addresses.
4. The storage device of claim 3 , further comprising a second processing unit connected to a messaging interface, the second processing unit being configured to:
receive, through the messaging interface, a data storage request comprising:
a block address and
data; and
in response to the receiving of the data storage request, store the data in a storage area in the storage media.
5. The storage device of claim 4 , wherein the second processing unit is further configured to:
receive, through the messaging interface, a data retrieval request comprising a block address;
in response to the receiving of the data retrieval request, retrieve data from a storage area in the storage media; and
send the retrieved data through the messaging interface.
6. The storage device of claim 5 , wherein the device data processing agent is configured to retrieve data from the storage area identified by a block address of the plurality of block addresses by sending a data retrieval request to the second processing unit through the messaging interface and receiving retrieved data through the messaging interface.
7. The storage device of claim 6 , wherein the messaging interface is a hardware mailbox connected to the first processing unit and the second processing unit, the hardware mailbox comprising a buffer to store data being sent to the second processing unit, and a circuit configured to interrupt the second processing unit when the buffer is not empty.
8. The storage device of claim 7 , wherein the first manifest further comprises a second binary comprising second instructions, and
wherein the device data processing agent is further configured to:
transfer the first binary to the container when the first binary is suitable for execution by the first processing unit, and
transfer the second binary to the container when the second binary is suitable for execution by the first processing unit.
9. A method for operating a storage device communicatively coupled to a host through a storage interface, the storage device being configured to store host data provided by the host through a virtual Transmission Control Protocol/Internet Protocol (TCP/IP) tunnel over the storage interface, the storage device comprising:
storage media; and
a first processing unit,
the method comprising:
instantiating, by the first processing unit through the virtual TCP/IP tunnel over the storage interface, within the storage device, a device data processing agent and a container,
receiving, by the device data processing agent through the virtual TCP/IP tunnel over the storage interface, a manifest from the host, the manifest comprising one or more binaries, each of the one or more binaries comprising instructions,
selecting, by the device data processing agent, a selected binary from the one or more binaries, the selected binary comprising selected instructions,
extracting the selected binary, from the manifest, within the storage device, and
executing the selected binary by the first processing unit to perform data processing on data stored in the storage device based on the selected instructions in the selected binary.
10. The method of claim 9 , wherein the method further comprises:
validating, by the container, a safety of the selected instructions, and
executing the selected instructions when the selected instructions are validated as safe.
11. The method of claim 10 , wherein the receiving, by the device data processing agent, of the manifest comprises receiving, by the device data processing agent, the manifest comprising:
a plurality of input file identifiers; and
a plurality of output file identifiers, and
the method further comprises:
sending, by the device data processing agent, a request to the host in the form of the plurality of input file identifiers and the plurality of output file identifiers through the storage interface; and
receiving, by the device data processing agent, a corresponding response from the host in the form of a plurality of block addresses through the storage interface, each block address identifying a storage area of the storage media.
12. The method of claim 11 , further comprising:
retrieving, by the device data processing agent, data from the storage area identified by a block address of the plurality of block addresses, and
transferring, by the device data processing agent, the retrieved data to the container.
13. The method of claim 11 , further comprising:
receiving data, by the device data processing agent, from the container; and
storing, by the device data processing agent, the received data in a storage area identified by a block address of the plurality of block addresses.
14. The method of claim 9 , wherein:
the one or more binaries comprise a first binary and a second binary of the one or more binaries, and
the selecting of a selected binary from the one or more binaries comprises:
selecting, by the device data processing agent, the first binary as the selected binary when the first binary is suitable for execution by the first processing unit, and
selecting, by the device data processing agent, the second binary as the selected binary when the second binary is suitable for execution by the first processing unit.