Method and system for condensed cache and acceleration layer integrated in servers

1. A method of operating a cache drive in a compute server of a computer cluster, the method comprising:

receiving, from an IO interface in the cache drive of the compute server, a write request to write data comprising a plurality of segments ordered randomly from a plurality of clients;

caching the data corresponding to the write request in a cache storage of the cache drive of the compute server;

performing one or more compute processes on the data, wherein the one or more compute processes include:

locating one or more segments from the plurality of segments, wherein the one or more segments are from one client of the plurality of clients; and

merging data from the one or more segments, to merge the data from the one client of the plurality of clients; and

in response to performing the one or more compute processes on the data, providing the processed data to a storage cluster for storing via the IO interface in the cache drive that is communicatively coupled to the storage cluster.

2. The method of claim 1 , wherein:

the one or more compute processes are performed by one or more processing units in the cache drive; and

the compute processes comprise cyclic redundancy checks, redundant array of independent disks encoding, or error correction code encoding.

3. The method of claim 1 , further comprising:

receiving, from the IO interface in the cache drive, a read request to read data from the storage cluster;

reading the data corresponding to the read request from the storage cluster via the IO interface in the cache drive; and

caching the data corresponding to the read request in the cache storage; and

providing the data cached in the cache drive to the computer cluster via the IO interface in the cache drive.

4. The method of claim 3 , further comprising:

providing the data cached in the cache drive to a plurality of other compute servers in the computer cluster.

5. The method of claim 3 , further comprising:

providing the data cached in the cache drive to a plurality of clients communicatively coupled to the cache drive.

6. The method of claim 3 , further comprising:

in response to receiving the read request, determining whether the data corresponding to the read request is cached in the cache storage; and

in response to a determination that the data corresponding to the read request is cached in the cache storage, providing the data cached in the cache drive to the computer cluster via the IO interface in the cache drive.

7. The method of claim 3 , further comprising:

in response to receiving the read request, determining potential data for data prefetching from the storage cluster; and

performing data prefetching on the potential data to cache the potential data in the cache storage.

8. The method of claim 1 , wherein the cache drive is communicatively coupled with the computer cluster and the storage cluster via a network card in the compute server.

9. The method of claim 1 , wherein:

the cache storage comprises one or more flash drives.

10. A non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a cache drive to cause the cache drive to perform a method, the method comprising:

receiving, from an IO interface in the cache drive of the compute server, a write request to write data comprising a plurality of segments ordered randomly from a plurality of clients;

caching the data corresponding to the write request in a cache storage of the cache drive of the compute server;

performing one or more compute processes on the data, wherein the one or more compute processes include:

locating one or more segments from the plurality of segments, wherein the one or more segments are from one client of the plurality of clients; and

merging data from the one or more segments, to merge the data from the one client of the plurality of clients; and

in response to performing the one or more compute processes on the data, providing the processed data to a storage cluster for storing via the IO interface in the cache drive that is communicatively coupled to the storage cluster.

11. A compute server in a computer cluster, the compute server comprising:

a cache drive, comprising:

a cache storage configured to store data;

an IO interface communicatively coupled to the computer cluster and a storage cluster; and

one or more processing units communicatively coupled to the cache storage and the IO interface, wherein the one or more processors are configured to cause the cache drive to:

receive, from the IO interface, a write request to write data comprising a plurality of segments ordered randomly from a plurality of clients;

cache the data corresponding to the write request in the cache storage;

perform one or more compute processes on the data, wherein the one or more compute processes include:

locating one or more segments from the plurality of segments, wherein the one or more segments are from one client of the plurality of clients; and

merging data from the one or more segments, to merge the data from the one client of the plurality of clients; and

in response to performing the one or more compute processes on the data, provide the processed data to the storage cluster for storing via the IO interface.

12. A cache drive in a compute server of a computer cluster, the cache drive comprising:

a cache storage configured to store data;

an IO interface communicatively coupled to the computer cluster and a storage cluster; and

one or more processing units communicatively coupled to the cache storage and the IO interface, wherein the one or more processors are configured to cause the cache drive to:

receive, from the IO interface, a write request to write data comprising a plurality of segments ordered randomly from a plurality of clients;

cache the data corresponding to the write request in the cache storage;

perform one or more compute processes on the data, wherein the one or more compute processes include:

locating one or more segments from the plurality of segments, wherein the one or more segments are from one client of the plurality of clients; and

merging data from the one or more segments, to merge the data from the one client of the plurality of clients; and

in response to performing the one or more compute processes on the data, provide the processed data to the storage cluster for storing via the IO interface.

13. The cache drive of claim 12 , wherein:

the compute processes comprise cyclic redundancy checks, redundant array of independent disks encoding, or error correction code encoding.

14. The cache drive of claim 12 , wherein the one or more processing units are further configured to cause the cache drive to:

receive, from the IO interface, a read request to read data from the storage cluster;

read the data corresponding to the read request from the storage cluster via the IO interface; and

cache the data corresponding to the read request in the cache storage; and

provide the data cached in the cache drive to the compute server computer cluster via the IO interface.

15. The cache drive of claim 14 , wherein the one or more processing units are further configured to cause the cache drive to:

in response to receiving the read request, determine whether the data corresponding to the read request is cached in the cache storage; and

in response to a determination that the data corresponding to the read request is cached in the cache storage, provide the data cached in the cache drive to the computer cluster via the IO interface.

16. The cache drive of claim 14 , wherein the one or more processing units are further configured to cause the cache drive to:

in response to receiving the read request, determine potential data for data prefetching from the storage cluster; and

perform data prefetching on the potential data to cache the potential data in the cache storage.

17. The cache drive of claim 12 , wherein the cache drive is communicatively coupled with the computer cluster and the storage cluster via a network card in the compute server.

18. The cache drive of claim 12 , wherein the cache storage comprises one or more flash drives.