Pattern-aware prefetching using parallel log-structured file system

1. A method for accessing at least a portion of a shared file, comprising:

performing the following steps, in response to at least one read request for at least a first portion of said shared file:

detecting at least one pattern in at least two consecutive non-sequential accesses of said shared file, wherein said at least one pattern is detected by analyzing distances between consecutive logical offsets in said at least one read request for said shared file; and

prefetching at least a second portion of said shared file from at least one storage device, wherein said second portion is predicted to be accessed based at least in part on said detected at least one pattern,

wherein said shared file is created by a plurality of processes, wherein each of said plurality of processes has a unique physical logfile for storing data written by a corresponding process and a unique physical index file for storing metadata for the corresponding process, wherein, for a given write operation of a data chunk to the shared file by a given one of said plurality of processes, (i) said data chunk is stored in said unique physical logfile corresponding to said given process, and (ii) an index entry is created for said unique physical index file corresponding to said given process, that maps said data chunk within a logical representation of said shared file to a location within said unique physical logfile, wherein a plurality of said index entries is stored in said unique physical index file corresponding to said given process in a compressed format using said detected at least one pattern and wherein said prefetching of said at least second portion of said shared file is performed using at least one of said unique physical index files, comprising said compressed format, for said plurality of processes and wherein the method is performed by at least one processing device comprising a processor coupled to a memory.

2. The method of claim 1 , wherein said prefetching step comprises a pre-allocation of said at least said second portion of said shared file.

3. The method of claim 1 , further comprising the step of storing said at least said second portion of said shared file in a prefetch cache.

4. The method of claim 3 , further comprising the step of providing said at least said second portion of said shared file from said prefetch cache to an application requesting said at least said second portion of said shared file.

5. The method of claim 4 , wherein a prefetch manager receives said detected at least one pattern from a pattern detector, predicts future requests based at least in part on said detected at least one pattern and requests a prefetch thread to fetch said at least said second portion of said shared file from a file system for storage in said prefetch cache.

6. The method of claim 4 , further comprising the step of determining whether said prefetch cache stores said second portion of said shared file requested in a read request.

7. The method of claim 1 , wherein said detecting step further comprises the steps of obtaining a plurality of read requests and detecting one or more patterns in said obtained plurality of read requests.

8. The method of claim 1 , wherein said unique physical index file indicates how said unique physical logfile for storing data written by the corresponding process was written and how said unique physical logfile for storing data written by the corresponding process is read.

9. A non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device to perform the following steps in response to at least one read request for at least a first portion of a shared file:

detecting, using at least one processing device of a pattern-aware prefetching file system, at least one pattern in at least two consecutive non-sequential accesses of said shared file, wherein said at least one pattern is detected by analyzing distances between consecutive logical offsets in said at least one read request for said shared file; and

prefetching at least a second portion of said shared file from at least one storage device, wherein said second portion is predicted to be accessed based at least in part on said detected at least one pattern,

wherein said shared file is created by a plurality of processes, wherein each of said plurality of processes has a unique physical logfile for storing data written by a corresponding process and a unique physical index file for storing metadata for the corresponding process, wherein, for a given write operation of a data chunk to the shared file by a given one of said plurality of processes, (i) said data chunk is stored in said unique physical logfile corresponding to said given process, and (ii) an index entry is created for said unique physical index file corresponding to said given process, that maps said data chunk within a logical representation of said shared file to a location within said unique physical logfile, wherein a plurality of said index entries is stored in said unique physical index file corresponding to said given process in a compressed format using said detected at least one pattern and wherein said prefetching of said at least second portion of said shared file is performed using at least one of said unique physical index files, comprising said compressed format, for said plurality of processes.

10. The non-transitory processor-readable storage medium of claim 9 , wherein said prefetch manager pre-allocates said at least said second portion of said shared file.

11. The non-transitory processor-readable storage medium of claim 9 , wherein said detecting step further comprises the steps of obtaining a plurality of read requests and detecting one or more patterns in said obtained plurality of read requests.

12. The non-transitory processor-readable storage medium of claim 9 , wherein said unique physical index file indicates how said unique physical logfile for storing data written by the corresponding process was written and how said unique physical logfile for storing data written by the corresponding process is read.

13. A pattern-aware prefetching file system, comprising:

a pattern detector of a pattern-aware prefetching file system for detecting, in response to at least one read request for at least a first portion of a shared file, using at least one processing device of said pattern-aware prefetching file system, at least one pattern in at least two consecutive non-sequential accesses of said shared file, wherein said at least one pattern is detected by analyzing distances between consecutive logical offsets in said at least one read request for said shared file; and

a prefetch manager for prefetching, in response to said at least one read request for said at least first portion of said shared file, at least a second portion of said shared file from at least one storage device, wherein said second portion is predicted to be accessed based at least in part on said detected at least one pattern; and

a prefetch cache for storing at least said portion of said shared file,

wherein said shared file is created by a plurality of processes, wherein each of said plurality of processes has a unique physical logfile for storing data written by a corresponding process and a unique physical index file for storing metadata for the corresponding process, wherein, for a given write operation of a data chunk to the shared file by a given one of said plurality of processes, (i) said data chunk is stored in said unique physical logfile corresponding to said given process, and (ii) an index entry is created for said unique physical index file corresponding to said given process, that maps said data chunk within a logical representation of said shared file to a location within said unique physical logfile, wherein a plurality of said index entries is stored in said unique physical index file corresponding to said given process in a compressed format using said detected at least one pattern and wherein said prefetching of said at least second portion of said shared file is performed using at least one of said unique physical index files, comprising said compressed format, for said plurality of processes.

14. The pattern-aware prefetching file system of claim 13 , wherein said prefetch manager pre-allocates said at least said second portion of said shared file.

15. The pattern-aware prefetching file system of claim 13 , wherein at least said second portion of said shared file from said prefetch cache are provided to an application requesting said at least said second portion of said shared file.

16. The pattern-aware prefetching file system of claim 15 , wherein said prefetch manager receives said detected at least one pattern from said pattern detector, predicts future requests based at least in part on said detected at least one pattern and requests a prefetch thread to fetch said at least said second portion of said shared file from another file system for storage in said prefetch cache.

17. The pattern-aware prefetching file system of claim 15 , wherein said prefetch manager determines whether said prefetch cache stores said second portion of said shared file requested in a read request.

18. The pattern-aware prefetching file system of claim 13 , wherein said pattern detector is further configured to obtain a plurality of read requests and detect one or more patterns in said obtained plurality of read requests.

19. The pattern-aware prefetching file system of claim 13 , wherein said a pattern detector is further configured to obtain a plurality of read requests.

20. The pattern-aware prefetching file system of claim 13 , wherein said unique physical index file indicates how said unique physical logfile for storing data written by the corresponding process was written and how said unique physical logfile for storing data written by the corresponding process is read.