Learning machine to optimize random access in a storage system
Mechanisms are provided for optimizing random access in a storage system. According to various embodiments, an access pattern may be identified for a plurality of data segments stored in a first arrangement on a storage medium. Each of the plurality of data segments may be stored at a respective first storage location on the storage medium in the first arrangement. The access pattern may indicate an order in which the data segments are likely to be retrieved from the storage medium. The plurality of data segments may be stored in a second arrangement on the storage medium based on the identified access pattern. Each of the plurality of data segments may be stored at a respective second storage location on the storage medium in the updated arrangement.
1. A method comprising:
identifying and recording an access pattern for a plurality of data segments stored in a first arrangement on a storage medium, the plurality of data segments including a designated set of data segments that have been deduplicated, each of the plurality of data segments being stored at a respective first storage location on the storage medium in the first arrangement, the identified access pattern indicating a sequence in which the data segments are likely to be retrieved from the storage medium, wherein recording the access pattern includes recording a frequency with which the access pattern has occurred, wherein recording the access pattern further includes identifying a designated content type associated with the plurality of data segments, the designated content type being one of a plurality of content types, and wherein the access pattern is identified at least in part based on at least a partial match with a typical access pattern associated with the designated content type; and
storing the plurality of data segments in a second arrangement on the storage medium based on the identified access pattern, each of the plurality of data segments being stored at a respective second storage location on the storage medium in the second arrangement, the respective first storage location being different than the respective second storage location for one or more of the plurality of data segments, wherein storing the plurality of data segments includes storing a duplicate copy of at least one of the designated set of data segments that have been deduplicated.
2. The method recited in claim 1 , wherein accessing the plurality of data segments substantially in accordance with the identified access pattern is faster under the second arrangement than under the first arrangement.
3. The method recited in claim 1 , wherein identifying the access pattern comprises:
analyzing first data access information describing one or more previous instances of retrieval of the plurality of data segments from the storage medium.
4. The method recited in claim 3 , wherein identifying the access pattern further comprises:
computing an estimate of the access pattern based on the first data access information.
5. The method recited in claim 4 , wherein identifying the access pattern further comprises:
analyzing second data access information describing one or more previous instances of retrieval of the plurality of data segments from the storage medium, and
computing an updated estimate of the access pattern based on the second data access information as part of an iterative learning process.
6. The method recited in claim 1 , wherein identifying the access pattern further comprises:
receiving predetermined access pattern information associated with the identified content type.
7. The method recited in claim 6 , wherein the predetermined access pattern information indicates one or more likely access patterns for the identified content types, the one or more likely access patterns including the identified access pattern.
8. The method recited in claim 1 , wherein storing the plurality of data segments in the second arrangement comprises:
moving each of the plurality of data segments from the respective first storage location to the respective second storage location when the respective first storage location is different than the respective second storage location.
9. A system comprising:
a storage module configured to store a plurality of data segments stored in a first arrangement, the plurality of data segments including a designated set of data segments that have been deduplicated, each of the plurality of data segments being stored at a respective first storage location on the storage module in the first arrangement; and
a processor configured to:
identify and record an access pattern for the plurality of data segments, the access pattern indicating a sequence in which the data segments are likely to be retrieved from the storage module, wherein recording the access pattern includes recording a frequency with which the access pattern has occurred, wherein recording the access pattern further includes identifying a designated content type associated with the plurality of data segments, the designated content type being one of a plurality of content types, and wherein the access pattern is identified at least in part based on at least a partial match with a typical access pattern associated with the designated content type, and
transmit an instruction to store the plurality of data segments in a second arrangement on the storage medium based on the identified access pattern, each of the plurality of data segments being stored at a respective second storage location on the storage medium in the second arrangement, the respective first storage location being different than the respective second storage location for one or more of the plurality of data segments, wherein storing the plurality of data segments includes storing a duplicate copy of at least one of the designated set of data segments that have been deduplicated.
10. The system recited in claim 9 , wherein accessing the plurality of data segments substantially in accordance with the identified access pattern is faster under the second arrangement than under the first arrangement.
11. The system recited in claim 9 , wherein identifying the access pattern comprises:
analyzing data access information describing one or more previous instances of retrieval of the plurality of data segments from the storage module.
12. The system recited in claim 11 , wherein identifying the access pattern further comprises:
computing an estimate of the access pattern based on the data access information.
13. The system recited in claim 9 , wherein identifying the access pattern further comprises:
analyzing second data access information describing one or more previous instances of retrieval of the plurality of data segments from the storage module, and
computing an updated estimate of the access pattern based on the second data access information as part of an iterative learning process.
14. The system recited in claim 9 , wherein identifying the access pattern further comprises:
receiving predetermined access pattern information associated with the identified content type.
15. The system recited in claim 14 , wherein the predetermined access pattern information indicates one or more likely access patterns for the identified content types, the one or more likely access patterns including the identified access pattern.
16. One or more non-transitory computer readable media having instructions stored thereon for performing a method, the method comprising:
identifying and recording an access pattern for a plurality of data segments stored in a first arrangement on a storage medium, the plurality of data segments including a designated set of data segments that have been deduplicated, each of the plurality of data segments being stored at a respective first storage location on the storage medium in the first arrangement, the identified access pattern indicating a sequence in which the data segments are likely to be retrieved from the storage medium, wherein recording the access pattern includes recording a frequency with which the access pattern has occurred, wherein recording the access pattern further includes identifying a designated content type associated with the plurality of data segments, the designated content type being one of a plurality of content types, and wherein the access pattern is identified at least in part based on at least a partial match with a typical access pattern associated with the designated content type; and
storing the plurality of data segments in a second arrangement on the storage medium based on the identified access pattern, each of the plurality of data segments being stored at a respective second storage location on the storage medium in the second arrangement, the respective first storage location being different than the respective second storage location for one or more of the plurality of data segments, wherein storing the plurality of data segments includes storing a duplicate copy of at least one of the designated set of data segments that have been deduplicated.
17. The one or more non-transitory computer readable media recited in claim 16 , wherein accessing the plurality of data segments substantially in accordance with the identified access pattern is faster under the second arrangement than under the first arrangement.