Method for efficient write path cache load to improve storage efficiency
According to some embodiments, a backup storage system receives a request from a client for writing a data segment associated with a file object stored to a storage system. In response to the request, the system writes the data segment to one of many storage units of the storage system. The system determines whether the data segment is associated with a file region of the file object that is frequently accessed. The system writes the data segment in a first of many of write-evict units (WEUs) stored in a solid state device (SSD) operating as a cache memory device for caching data, after it is determined that the data segment is associated with the frequently accessed file region.
1. A computer-implemented method for accessing data segments stored in a storage system, the method comprising:
receiving a request from a client for writing a data segment associated with a file object stored to a storage system;
in response to the request, writing the data segment to one of a plurality of storage units of the storage system;
determining whether the data segment is associated with a frequently accessed file region of the file object, the file object comprising a plurality of file regions; and
in addition to writing the data segment to one of the plurality of storage units of the storage system, writing the data segment in a first of a plurality of write-evict units (WEUs) stored in a solid state device (SSD) operating as a cache memory device for caching data, in response to determining that the data segment is associated with the frequently accessed file region, each of the plurality of WEUs comprising a plurality of independently accessible extents.
2. The method of claim 1 , further comprising not writing the data segment to the SSD device in response to determining that the data segment is not associated with the frequently accessed file region.
3. The method of claim 1 , further comprising adding an index entry into a cache index to map a fingerprint of the data segment to a storage location of the first WEU in the SSD device.
4. The method of claim 1 , further comprising maintaining a histogram for the file object recording access statistics of the file object in a predetermined time period in the past, wherein whether the data segment is frequently accessed is determined based on the histogram of the file object.
5. The method of claim 1 , further comprising:
examining a data stream associated with the data segment to determine an access pattern of the data stream; and
determining whether the data segment is randomly accessed based on the accessed pattern of the data stream, wherein the data segment is cached in the SSD device only if the data segment is randomly accessed.
6. The method of claim 1 , further comprising:
in response to a second request to read a second segment, determining whether the SSD device stores the second segment;
retrieving the second data segment from a second of the storage units if the SSD device does not contain the second data segment; and
caching the second data segment in the SSD device if the second data segment is associated with a second file region that is frequently accessed.
7. The method of claim 1 , wherein the data segment is a part of incremental backup representing a difference between a first snapshot and a second snapshot.
8. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations to sort data elements in a data array, the operations comprising:
receiving a request from a client for writing a data segment associated with a file object stored to a storage system;
in response to the request, writing the data segment to one of a plurality of storage units of the storage system;
determining whether the data segment is associated with a frequently accessed file region of the file object, the file object comprising a plurality of file regions; and
in addition to writing the data segment to one of the plurality of storage units of the storage system, writing the data segment in a first of a plurality of write-evict units (WEUs) stored in a solid state device (SSD) operating as a cache memory device for caching data, in response to determining that the data segment is associated with the frequently accessed file region, each of the plurality of WEUs comprising a plurality of independently accessible extents.
9. The non-transitory machine-readable medium of claim 8 , further comprising not writing the data segment to the SSD device in response to determining that the data segment is not associated with the frequently accessed file region.
10. The non-transitory machine-readable medium of claim 8 , further comprising adding an index entry into a cache index to map a fingerprint of the data segment to a storage location of the first WEU in the SSD device.
11. The non-transitory machine-readable medium of claim 8 , further comprising maintaining a histogram for the file object recording access statistics of the file object in a predetermined time period in the past, wherein whether the data segment is frequently accessed is determined based on the histogram of the file object.
12. The non-transitory machine-readable medium of claim 8 , further comprising:
examining a data stream associated with the data segment to determine an access pattern of the data stream; and
determining whether the data segment is randomly accessed based on the accessed pattern of the data stream, wherein the data segment is cached in the SSD device only if the data segment is randomly accessed.
13. The non-transitory machine-readable medium of claim 8 , further comprising:
in response to a second request to read a second segment, determining whether the SSD device stores the second segment;
retrieving the second data segment from a second of the storage units if the SSD device does not contain the second data segment; and
caching the second data segment in the SSD device if the second data segment is associated with a second file region that is frequently accessed.
14. The non-transitory machine-readable medium of claim 8 , wherein the data segment is a part of incremental backup representing a difference between a first snapshot and a second snapshot.
15. A data processing system, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including
receiving a request from a client for writing a data segment associated with a file object stored to a storage system;
in response to the request, writing the data segment to one of a plurality of storage units of the storage system;
determining whether the data segment is associated with a frequently accessed file region of the file object, the file object comprising a plurality of file regions; and
in addition to writing the data segment to one of the plurality of storage units of the storage system, writing the data segment in a first of a plurality of write-evict units (WEUs) stored in a solid state device (SSD) operating as a cache memory device for caching data, in response to determining that the data segment is associated with the frequently accessed file region, each of the plurality of WEUs comprising a plurality of independently accessible extents.
16. The data processing system of claim 15 , further comprising not writing the data segment to the SSD device in response to determining that the data segment is not associated with the frequently accessed file region.
17. The data processing system of claim 15 , further comprising adding an index entry into a cache index to map a fingerprint of the data segment to a storage location of the first WEU in the SSD device.
18. The data processing system of claim 15 , further comprising maintaining a histogram for the file object recording access statistics of the file object in a predetermined time period in the past, wherein whether the data segment is frequently accessed is determined based on the histogram of the file object.
19. The data processing system of claim 15 , further comprising:
examining a data stream associated with the data segment to determine an access pattern of the data stream; and
determining whether the data segment is randomly accessed based on the accessed pattern of the data stream, wherein the data segment is cached in the SSD device only if the data segment is randomly accessed.
20. The data processing system of claim 15 , further comprising:
in response to a second request to read a second segment, determining whether the SSD device stores the second segment;
retrieving the second data segment from a second of the storage units if the SSD device does not contain the second data segment; and
caching the second data segment in the SSD device if the second data segment is associated with a second file region that is frequently accessed.
21. The data processing system of claim 15 , wherein the data segment is a part of incremental backup representing a difference between a first snapshot and a second snapshot.