Method and system for enhancing throughput of big data analysis in a NAND-based read source storage

1. A computer-implemented method for facilitating data access in a storage device, the method comprising:

obtaining, by a storage device, a file from an original physical media separate from the storage device, wherein the file comprises compressed data which has been previously encoded based on an error correction code;

storing, on a physical media of the storage device, the obtained file as a read-only replica;

receiving, by the storage device, a request to read the file stored on the physical media of the storage device; and

decoding, by the storage device based on the error correction code (ECC), the file to obtain ECC-decoded data,

wherein the ECC-decoded data is subsequently decompressed by a computing device associated with the storage device and returned as the requested file.

2. The method of claim 1 , wherein the request to read the file is received from a requesting entity based on a first protocol, and wherein subsequent to the computing device decompressing the ECC-decoded data, the method further comprises:

returning, to the requesting entity, the decompressed data as the requested file without performing any ECC encoding.

3. The method of claim 1 , wherein decompressing the ECC-decoded data is performed by parallel decompression engines of the computing device.

4. The method of claim 1 , wherein the request to read the file is received by the computing device and transmitted by the computing device to the storage device via a system which comprises at least one of each of:

a first Ethernet switch and a second Ethernet switch;

a first smart network interface card (NIC) and a second smart network interface card (NIC); and

a plurality of Peripheral Interconnect Express (PCIe) switches,

wherein the first smart NIC and the second smart NIC each comprise a simple storage node which includes:

an uplink to the first Ethernet switch and the second Ethernet switch; and

a downlink to the plurality of PCIe switches via a plurality of PCIe lanes which are used to connect to the storage device and a plurality of other storage devices.

5. The method of claim 4 , wherein the request to read the file is further received by a second computing device which is a backup or a high-availability server for the computing device and is further transmitted by the second computing device to the storage device via the system.

6. The method of claim 1 , further comprising:

storing, by the computing device, a mapping of a name of the file to a current physical block address associated with the storage device; and

in response to detecting a condition to move the file from the current physical block address to a new physical block address associated with the storage device, updating the mapping of the name of the file to the new physical block address.

7. The method of claim 1 , wherein a format for the file stored in the storage device includes one or more:

a pre-amble which denotes a starting location associated with the file;

a unique file identifier for the file;

a content of the file;

a post-amble which denotes an ending location associated with the file; and

a cyclic redundancy check (CRC) signature for verification of consistency of the content of the file.

8. The method of claim 1 , wherein the storage device does not comprise:

a module or unit which performs ECC-encoding;

a dynamic random access memory (DRAM) interface and DRAM modules accessed via the DRAM interface; and

a processor which performs flash translation layer (FTL) functionality, including mapping a logical block address to a physical block address.

9. A computer system for facilitating data access in a system, the system comprising:

a processor; and

a memory coupled to the processor and storing instructions, which when executed by the processor cause the processor to perform a method, wherein the computer system comprises a storage device, the method comprising:

obtaining, by a storage device, a file from an original physical media separate from the storage device, wherein the file comprises compressed data which has been previously encoded based on an error correction code;

storing, a physical media of the storage device, the obtained file as a read-only replica;

receiving, by the storage device, a request to read the file stored on the physical media of the storage device; and

decoding, by the storage device based on the error correction code (ECC), the file to obtain ECC-decoded data,

wherein the ECC-decoded data is subsequently decompressed by a computing device associated with the storage device and returned as the requested file.

10. The computer system of claim 9 , wherein the request to read the file is received from a requesting entity based on a first protocol, and wherein subsequent to the computing device decompressing the ECC-decoded data, the method further comprises:

returning, to the requesting entity, the decompressed data as the requested file without performing any ECC encoding.

11. The computer system of claim 9 , wherein decompressing the ECC-decoded data is performed by parallel decompression engines of the computing device.

12. The computer system of claim 9 , wherein the request to read the file is received by the computing device and transmitted by the computing device to the storage device via a system which comprises at least one of each of:

a first Ethernet switch and a second Ethernet switch;

a first smart network interface card (NIC) and a second smart network interface card (NIC); and

a plurality of Peripheral Interconnect Express (PCIe) switches,

wherein the first smart NIC and the second smart NIC each comprise a simple storage node which includes:

an uplink to the first Ethernet switch and the second Ethernet switch; and

a downlink to the plurality of PCIe switches via a plurality of PCIe lanes which are used to connect to the storage device and a plurality of other storage devices.

13. The computer system of claim 12 , wherein the request to read the file is further received by a second computing device which is a backup or a high-availability server for the computing device and is further transmitted by the second computing device to the storage device via the system.

14. The computer system of claim 9 , wherein the method further comprises:

storing, by the computing device, a mapping of a name of the file to a current physical block address associated with the storage device; and

in response to detecting a condition to move the file from the current physical block address to a new physical block address associated with the storage device, updating the mapping of the name of the file to the new physical block address.

15. The computer system of claim 9 , wherein a format for the file stored in the storage device includes one or more:

a pre-amble which denotes a starting location associated with the file;

a unique file identifier for the file;

a content of the file;

a post-amble which denotes an ending location associated with the file; and

a cyclic redundancy check (CRC) signature for verification of consistency of the content of the file.

16. The computer system of claim 9 , wherein the storage device does not comprise:

a module or unit which performs ECC-encoding;

a dynamic random access memory (DRAM) interface and DRAM modules accessed via the DRAM interface; and

a processor which performs flash translation layer (FTL) functionality, including mapping a logical block address to a physical block address.

17. A distributed storage system for facilitating one or more big data analysis applications through one or more clients, the distributed storage system comprising:

a front-end head server;

a switch; and

a plurality of NAND cards coupled to the switch,

wherein a NAND card is configured to:

obtain a file from an original physical media separate from the storage device, wherein the file comprises compressed data which has been previously encoded based on an error correction code;

store on a physical media of the NAND card, the obtained file as a read-only replica;

receive, via the switch, a request to read the file stored on the NAND card; and

decode, by the NAND card based on the error correction code (ECC), the file to obtain ECC-decoded data, and

wherein the front-end head server is configured to decompress the ECC-decoded data and return the decompressed data as the requested file.

18. The distributed storage system of claim 17 ,

wherein the front-end head server is configured to receive the request to read the file,

wherein the NAND card is further configured to:

return the ECC-decoded data to the front-end head server without performing any ECC encoding; and

wherein the NAND card does not comprise:

a module or unit which performs ECC-encoding;

a dynamic random access memory (DRAM) interface and DRAM modules accessed via the DRAM interface; and

a processor which performs flash translation layer (FTL) functionality, including mapping a logical block address to a physical block address,

thereby facilitating the big data analysis applications to efficiently analyze data by performing a reduced amount of ECC encoding.