METHODS AND SYSTEMS FOR STORING DATA SEQUENTIALLY ON A MEDIA
Systems and methods for storing data are provided herein using a data archive architecture controlled by a head server. The head server can write data in a contiguous manner across multiple storage devices by partitioning objects into multiple portions and spreading the portions across the multiple storage devices. Portions of a first object can be transmitted to a first subset of storage devices and portions of a second object can be transmitted to a second subset of storage devices. The first subset of storage devices can write the portions of the first object to a first write location indicated by a write pointer. The head server can increment the write pointer based on a length of the portions of the first object to a second write location. The second subset of storage devices can write the portions of the second object to the second write location.
1 . A method to store data sequentially on a media comprising:
receiving, by a head server in communication with a plurality of storage devices, a first object;
dividing, by the head server, the first object into a first portion having a first length and a second portion having a second length, the first length different from the second length;
transmitting, by the head server, the first portion to a first storage device of the plurality of storage devices;
writing, by the first storage device, the first portion to a first write location of the first storage device indicated by a write pointer;
incrementing, by the head server, the write pointer to a second write location based on the first length, the second write location of the first storage device;
transmitting, by the head server, a first subset of the second portion to the first storage device and a second subset of the second portion to a second storage device of the plurality of storage devices;
writing, by the first storage device, the first subset of the second portion to the second write location of the first storage device, the second write location in a contiguous order indicated by a write pointer;
incrementing, by the head server, the write pointer to a third write location of the second storage device; and
writing, by the second storage device, the second subset of the second portion to the third write location of the second storage device, the first portion and the second portion stored in a contiguous order from the first storage device to the second storage device.
2 . The method of claim 1 , further comprising:
determining a first subset length of the first subset of the second portion and a second subset length of the second subset of the second portion; and
incrementing, by the head server, the write pointer to the third write location based on the first subset length of the first subset of the second portion.
3 . The method of claim 1 , further comprising assigning write locations to each of the storage devices of the plurality of storage devices in a contiguous order.
4 . The method of claim 1 , further comprising organizing the first storage device and the second storage device in a contiguous track in the plurality of storage devices.
5 . The method of claim 1 , further comprising determining storage sizes for each of the storage devices of the plurality of storage devices.
6 . The method of claim 1 , further comprising determining the first storage device has a first storage size and the second storage device has a second storage size, the first storage size different from the second storage size.
7 . The method of claim 1 , further comprising:
extracting metadata from the first object; and
storing the metadata corresponding to the first object to a storage media independent from the plurality storage devices.
8 . The method of claim 1 , further comprising storing the first write location corresponding to the first portion, the second write location corresponding to the first subset of the second portion, and the third write location corresponding to the second subset of the second portion to a storage media independent from the plurality storage devices.
9 . The method of claim 1 , further comprising writing the first portion and the second portion with disk media using shingled magnetic recording (SMR) disk drives.
10 . The method of claim 1 , further comprising determining the first write location of the first storage device using a first hash calculation.
11 . The method of claim 1 , further comprising:
generating an encryption key for the first object;
encrypting the first object with the encryption key; and
recording the encryption key with the first write location.
12 . A system to store data sequentially on a media, comprising:
a head server in communication with a plurality of storage devices, the head server configured to:
receive a first object;
divide the first object into a first portion having a first length and a second portion having a second length, the first length different from the second length;
transmit the first portion to a first storage device of the plurality of storage devices indicated by a first write location at the first storage device;
increment a write pointer to a second write location based on the first length, the second write location of the first storage device; and
transmit a first subset of the second portion to the first storage device and a second subset of the second portion to a second storage device of the plurality of storage devices;
increment the write pointer to a third write location of the second storage device; and
the first storage device configured to:
write the first portion to the first write location of the first storage device; and
write the first subset of the second portion to the second write location of the first storage device; and
the second storage device configured to:
write the second subset of the second portion to the third write location of the second storage device, the first portion and the second portion stored in a contiguous order from the first storage device to the second storage device.
13 . The system of claim 12 , wherein the head server is further configured to:
determine a first subset length of the first subset of the second portion and a second subset length of the second subset of the second portion; and
increment the write pointer to the third write location based on the first subset length of the first subset of the second portion.
14 . The system of claim 12 , wherein the head server is further configured to assign write locations to each of the storage devices of the plurality of storage devices in a contiguous order.
15 . The system of claim 12 , wherein the head server is further configured to organize the first storage device and the second storage device in a contiguous track in the plurality of storage devices.
16 . The system of claim 12 , wherein the head server is further configured to determine storage sizes for each of the storage devices of the plurality of storage devices.
17 . The system of claim 12 , wherein the head server is further configured to determine the first storage device has a first storage size and the second storage device has a second storage size, the first storage size different from the second storage size.
18 . The system of claim 12 , wherein the head server is further configured to:
extract metadata from the first object; and
store the metadata corresponding to the first object to a storage media independent from the plurality storage devices.
19 . The system of claim 12 , wherein the head server is further configured to:
store the first write location corresponding to the first portion, the second write location corresponding to the first subset of the second portion, and the third write location corresponding to the second subset of the second portion to a storage media independent from the plurality storage devices.
20 . The system of claim 12 , wherein the head server is further configured to:
generate an encryption key for the first object;
encrypt the first object with the encryption key; and
record the encryption key with the first write location.