Systems and methods for random to sequential storage mapping
A data storage system includes a first storage layer, a second storage layer, an I/O manager, and a data organizer. The first storage layer utilizes a first type of data storage device. The first storage layer includes (i) a first data bucket that includes first data having a first data attribute, the first data bucket including a first data limit, and (ii) a second data bucket. The second storage layer utilizes a second type of data storage device. The I/O manager receives a data write request from the user and directs the data write request to the first storage layer. The data organizer (a) determines whether data in the data write request has the first data attribute; and (b) stores the data in the data write request in at least one of the first data bucket and the second data bucket if the data in the data write request has the first data attribute. When a volume of data in the first data bucket reaches the first data limit, the data organizer moves the volume of data in the first data bucket from the first storage layer to the second storage layer.
1. A data storage system for use by a user, the data storage system comprising:
a first storage layer that utilizes a first type of data storage device, the first storage layer including (i) a first data bucket that includes first data having a first data attribute, the first data bucket including a first data limit regarding a maximum data volume that can be retained within the first data bucket, and (ii) a second data bucket;
a second storage layer that utilizes a second type of data storage device that is different than the first type of data storage device;
an I/O manager that is configured to receive a data write request from the user and to direct the data write request to the first storage layer; and
a data organizer that is configured to (a) determine whether data in the data write request has the first data attribute; and (b) store the data in the data write request in at least one of the first data bucket and the second data bucket if the data in the data write request has the first data attribute; and
wherein when a volume of data in the first data bucket reaches the first data limit, the data organizer is further configured to move the volume of data in the first data bucket from the first storage layer to the second storage layer.
2. The data storage system of claim 1 wherein, if the data in the data write request does not have the first data attribute, the data organizer is further configured to create a third data bucket within the first storage layer, and store the data in the data write request in the third data bucket.
3. The data storage system of claim 1 wherein the first data attribute includes at least one of a data tag, a data policy and metadata that is common for all of the first data in the first data bucket.
4. The data storage system of claim 1 wherein the first data attribute includes one of a data lifetime policy, a data priority policy, a data protection policy, and a temporal locality that is common for all of the first data in the first data bucket.
5. The data storage system of claim 1 wherein the first type of data storage device is a random-access data storage device.
6. The data storage system of claim 5 wherein the first type of data storage device includes a solid-state drive.
7. The data storage system of claim 5 wherein the first type of data storage device includes a flash storage device.
8. The data storage system of claim 5 wherein the second type of data storage device is one of a sequential data storage device and a semi-sequential data storage device.
9. The data storage system of claim 8 wherein the second type of data storage device includes a shingled magnetic recording device.
10. The data storage system of claim 1 further comprising a segment manager that is configured to search the second storage layer for available storage space.
11. The data storage system of claim 10 wherein the segment manager is further configured to maintain a record of the available storage space on the second storage layer.
12. A method for storing data for a user, the method comprising the steps of:
receiving a data write request from the user with an I/O manager;
directing the data write request to a first storage layer with the I/O manager, the first storage layer utilizing a first type of data storage device, the first storage layer including (i) a first data bucket that includes first data having a first data attribute, the first data bucket including a first data limit regarding a maximum data volume that can be retained within the first data bucket, and (ii) a second data bucket;
determining whether data in the data write request has the first data attribute with a data organizer;
storing the data in the data write request in at least one of the first data bucket and the second data bucket with the data organizer if the data in the data write request has the first data attribute; and
when a volume of data in the first data bucket reaches the first data limit, moving the volume of data in the first data bucket from the first storage layer to a second storage layer with the data organizer, the second storage layer utilizing a second type of data storage device that is different than the first type of data storage device.
13. The method of claim 12 wherein, if the data in the data write request does not have the first data attribute, the method further comprises the steps of creating a third data bucket within the first storage layer with the data organizer;
and storing the data in the data write request in the third data bucket with the data organizer.
14. The method of claim 12 wherein the step of directing includes the first data attribute including at least one of a data tag, a data policy and metadata that is common for all of the first data in the first data bucket.
15. The method of claim 12 wherein the step of directing includes the first attribute including one of a data lifetime policy, a data priority policy, a data protection policy and a temporal locality that is common for all of the first data in the first data bucket.
16. The method of claim 12 wherein the step of directing includes the first type of data storage device being a random-access data storage device.
17. The method of claim 16 wherein the step of moving the volume of data includes the second type of data storage device being one of a sequential data storage device and a semi-sequential data storage device.
18. A data handling system for use by a user, the data handling system comprising:
an I/O manager that is configured to receive a data write request from the user and to direct the data write request to a first storage layer that utilizes a first type of data storage device, the first storage layer including (i) a first data bucket that includes first data having a first data attribute, the first data bucket including a first data limit regarding a maximum data volume that can be retained within the first data bucket, and (ii) a second data bucket; and
a data organizer that is configured to (a) determine whether data in the data write request has the first data attribute; and (b) store the data in the data write request in at least one of the first data bucket and the second data bucket if the data in the data write request has the first data attribute; and
wherein when a volume of data in the first data bucket reaches the first data limit, the data organizer is further configured to move the volume of data in the first data bucket from the first storage layer to a segment of a second storage layer having a segment size that substantially corresponds to the first data limit, the second storage layer utilizing a second type of data storage device that is different than the first type of data storage device.
19. The data handling system of claim 18 wherein, if the data in the data write request does not have the first data attribute, the data organizer is further configured to create a third data bucket within the first storage layer, and store the data in the data write request in the third data bucket.
20. The data handling system of claim 18 wherein the first type of data storage device is a random-access data storage device; and wherein the second type of data storage device is one of a sequential data storage device and a semi-sequential data storage device.