Implementing read/write, multi-versioned file system on top of backup data
Various embodiments of a computer system and methods are disclosed. In one embodiment, a computer system includes hosts coupled to a backup medium. The backup medium stores data segments and objects, each object referencing one or more segments. A second host conveys a write request to a first host identifying a data entity comprising segments referenced by a first object. The first host restores a copy of the data entity and creates a second object referencing the segments that were referenced by the first object in response to receiving the write request. The first host adds segment references to the second object in response to receiving a change that adds data to the copy or deletes a portion of a segment from the copy. The first host removes a segment reference from the second object if a change deletes all of a segment from the copy.
1. A computer system comprising:
a first host configured to store a plurality of backed up files on a backup medium, each file of the plurality of backed up files comprising one or more of a plurality of data segments, wherein in said storing the plurality of backed up files the first host is configured to:
store the plurality of data segments on the backup medium; and
for each respective file of the plurality of backed up files, store on the backup medium a respective data object that references the one or more data segments of the respective file;
a backup catalog configured to store metadata describing the plurality of backed up files, wherein the metadata includes a separate entry for each of the backed up files, and wherein the entry for each respective file includes a filename that corresponds to the respective file and a pointer to the respective data object that references the one or more data segments of the respective file; and
a second host coupled to the first host via a network, wherein the second host comprises a virtual file system interface to the backup medium, and wherein the second host is configured to convey to the first host, via the virtual file system interface, a request to open a first backed up file of the plurality of backed up files;
wherein in response to receiving the request to open the first backed up file, the first host is configured to:
create a new data object on the backup medium corresponding to a new version of the first backed up file;
add pointers to the new data object to reference the one or more data segments of the first backed up file; and
add a new entry to the backup catalog that identifies the new version of the first backed up file, wherein the new entry includes a pointer to the new data object;
wherein the second host is further configured to: after said conveying the request to open the first backed up file, convey to the first host, via the virtual file system interface, one or more modification requests to modify the first backed up file;
wherein in response to receiving the one or more modification requests, the first host is configured to:
store one or more new data segments on the backup medium; and
modify the new data object to reference the one or more new data segments, wherein in said modifying the new data object the first host is configured to perform one or more of: add one or more new pointers to the new data object to reference the one or more new data segments; or remove one or more of the previously existing pointers from the new data object.
2. The system as recited in claim 1 , wherein the first host is configured to utilize a single-instance storage technique whereby data is de-duplicated to remove redundant copies on the backup medium.
3. The system as recited in claim 1 , wherein the virtual file system interface is configured to enable the second host to create and store multiple versions of the backed up files on the backup medium.
4. The system as recited in claim 3 , wherein the catalog is further configured to store metadata describing branches containing multiple versions of the plurality of backed up files.
5. The system as recited in claim 1 ,
wherein the second host is further configured to: after said conveying the one or more modification requests, convey to the first host, via the virtual file system interface, a request to close the first file;
wherein in response to receiving the request to close the first file, the first host is configured to update one or more attributes stored in the new entry that was added to the backup catalog.
6. A method for accessing backup data, the method comprising:
storing a plurality of backed up files on a backup medium coupled to a first host, each file of the plurality of files comprising one or more of a plurality of data segments, wherein said storing the plurality of backed up files includes:
storing the plurality of data segments on the backup medium; and
for each respective file of the plurality of backed up files, storing on the backup medium a respective data object that references the one or more data segments of the respective file;
storing a backup catalog including metadata describing the plurality of backed up files, wherein the metadata includes a separate entry for each of the backed up files, and wherein the entry for each respective file includes a filename that corresponds to the respective file and a pointer to the respective data object that references the one or more data segments of the respective file;
providing a virtual file system interface on a second host to access the backup medium;
conveying from the second host to the first host, via the virtual file system interface, a request to open a first backed up file of the plurality of backed up files;
in response to the first host receiving the request to open the first backed up file:
creating a new data object on the backup medium corresponding to a new version of the first backed up file;
adding pointers to the new data object to reference the one or more data segments of the first backed up file; and
adding a new entry to the backup catalog that identifies the new version of the first backed up file, wherein the new entry includes a pointer to the new data object;
after said conveying the one or more modification requests, conveying from the second host to the first host, via the virtual file system interface, one or more modification requests to modify the first backed up file;
in response to the first host receiving the one or more modification requests:
storing one or more new data segments on the backup medium; and
modifying the new data object to reference the one or more new data segments, wherein said modifying the new data object includes one or more of: adding one or more new pointers to the new data object to reference the one or more new data segments; or removing one or more of the previously existing pointers from the new data object.
7. The method as recited in claim 6 , further comprising utilizing a single-instance storage technique on the backup medium whereby data is de-duplicated to remove redundant copies.
8. The system as recited in claim 6 , further comprising creating and storing multiple versions of the backed up files on the backup medium through the virtual file system.
9. The method as recited in claim 8 , further comprising storing metadata describing branches containing multiple versions of the plurality of the backed up files in the catalog.
10. The method as recited in claim 6 , further comprising:
after said conveying the one or more modification requests, conveying from the second host to the first host, via the virtual file system interface, a request to close the first file;
in response to the first host receiving the request to close the first file, updating one or more attributes stored in the new entry that was added to the backup catalog.
11. A plurality of non-transitory computer readable media storing computer instructions that are executable by one or more processors to:
store a plurality of backed up files on a backup medium coupled to a first host, each file of the plurality of files comprising one or more of a plurality of data segments, wherein said storing the plurality of backed up files includes:
storing the plurality of data segments on the backup medium; and
for each respective file of the plurality of backed up files, storing on the backup medium a respective data object that references the one or more data segments of the respective file;
store a backup catalog including metadata describing the plurality of backed up files, wherein the metadata includes a separate entry for each of the backed up files, and wherein the entry for each respective file includes a filename that corresponds to the respective file and a pointer to the respective data object that references the one or more data segments of the respective file;
provide a virtual file system interface on a second host to access the backup medium;
convey from the second host to the first host, via the virtual file system interface, a request to open a first backed up file of the plurality of backed up files;
in response to the first host receiving the request to open the first backed up file:
create a new data object on the backup medium corresponding to a new version of the first backed up file;
add pointers to the new data object to reference the one or more data segments of the first backed up file; and
add a new entry to the backup catalog that identifies the new version of the first backed up file, wherein the new entry includes a pointer to the new data object;
after said conveying the one or more modification requests, convey from the second host to the first host, via the virtual file system interface, one or more modification requests to modify the first backed up file;
in response to the first host receiving the one or more modification requests:
store one or more new data segments on the backup medium; and
modify the new data object to reference the one or more new data segments, wherein said modifying the new data object includes one or more of: adding one or more new pointers to the new data object to reference the one or more new data segments; or removing one or more of the previously existing pointers from the new data object.
12. The computer readable medium as recited in claim 11 , wherein the computer instructions are executable by the one or more processors to cause the first host to utilize a single-instance storage technique whereby data is de-duplicated to remove redundant copies on the backup medium.
13. The computer readable medium as recited in claim 11 , wherein the instructions are further executable to create and store multiple versions of the backed up files on the backup medium through the virtual file system.
14. The computer readable medium as recited in claim 11 , wherein the computer instructions are further executable by the one or more processors to:
after said conveying the one or more modification requests, convey from the second host to the first host, via the virtual file system interface, a request to close the first file;
in response to the first host receiving the request to close the first file, update one or more attributes stored in the new entry that was added to the backup catalog.