Data storage system with rapid restore capability
An improved information management system that implements a staging area or cache to temporarily store primary data in a native format before the primary data is converted into secondary copies in a secondary format is described herein. For example, the improved information management system can include various media agents that each include one or more high speed drives. When a client computing device provides primary data for conversion into secondary copies, the primary data can initially be stored in the native format in the high speed drive(s). If the client computing device then submits a request for the primary data, the media agent can simply retrieve the primary data from the high speed drive(s) and transmit the primary data to the client computing device. Because the primary data is already in the native format, no conversion operations are performed by the media agent, thereby reducing the restore delay.
1. A computer-implemented method performed by a system that comprises: a first computing device in communication with a client computing device that is configured to execute an application that generates a first data file in a native format, wherein the native format is associated with the application;
wherein the system further comprises a first drive that comprises one or more first data storage devices; and
wherein the system further comprises a second drive that comprises one or more second data storage devices, wherein the first drive is configured to read the first data file from one or more of the one or more first data storage devices faster than the second drive is configured to read the first data file from one or more of the one or more second data storage devices; and
wherein the computer-implemented method comprises:
receiving, from the client computing device, the first data file in the native format;
storing the first data file in the native format in the first drive;
after a first amount of time following the storing of the first data file in the first drive, moving the first data file in the native format from the first drive to the second drive;
after a second amount of time that is longer than the first amount of time:
(a) converting the first data file from the native format into one or more secondary copies that are in a secondary copy format, wherein the secondary copy format is distinct from the native format, and
(b) storing the one or more secondary copies, in the secondary copy format, in the second drive;
receiving a request for at least part of the first data file; and
responsive to the request, depending on a timing of the request compared to the first amount of time and the second amount of time, serve at least part of the first data file from one of: the first drive and the second drive.
2. The computer-implemented method of claim 1 further comprising:
after a third amount of time that is longer than the second amount of time, storing the one or more secondary copies, in the secondary copy format, at a secondary storage device that is distinct from and in communication with the first computing device; and
responsive to the request received after the one or more secondary copies are stored at the secondary storage device, (A) restoring the one or more secondary copies from the secondary copy format into the first data file in the native format at one of: the first drive and the second drive, and (B) transmitting at least part of the first data file as restored in the native format.
3. The computer-implemented method of claim 1 , further comprising:
for storage at one or more of the first drive and the second drive, dividing the first data file received from the client computing device into a plurality of file extents that each have a smaller size than the first data file; and
responsive to the request, serving a first file extent among the plurality of file extents.
4. The computer-implemented method of claim 1 , further comprising: based on the request being received before the first data file has been moved to the second drive, transmitting at least part of the first data file in the native format from the first drive.
5. The computer-implemented method of claim 1 , further comprising: based on the request being received after the first data file has been moved to the second drive and before the first data file has been converted into the one or more secondary copies, transmitting at least part of the first data file in the native format from the second drive.
6. The computer-implemented method of claim 1 , further comprising: based on the request being received after the first data file has been converted into the one or more secondary copies stored at the second drive, (A) restoring the one or more secondary copies from the secondary copy format into the first data file in the native format, and (B) transmitting at least part of the first data file in the native format from the second drive.
7. The computer-implemented method of claim 1 , wherein a storage manager that executes on a third computing device is configured to manage one or more storage policies that define the first amount of time and the second amount of time, wherein the third computing device is distinct from one or more of: the client computing device and the first computing device, and wherein the third computing device comprises one or more hardware processors.
8. The computer-implemented method of claim 7 , wherein the one or more storage policies further define when to store the one or more secondary copies, in the secondary copy format, at a secondary storage device that is distinct from and in communication with the first computing device.
9. The computer-implemented method of claim 1 , wherein the first computing device is implemented as a cloud computing resource in a cloud computing environment.
10. The computer-implemented method of claim 2 , wherein one or more of the first drive, the second drive, and the secondary storage device is implemented as a cloud storage resource in a cloud storage environment.
11. A system comprising:
a first computing device in communication with a client computing device that comprises one or more hardware processors, wherein the client computing device is configured to execute an application that generates a first data file in a native format, wherein the native format is associated with the application;
a first drive that comprises one or more first data storage devices; and
a second drive that comprises one or more second data storage devices, wherein the first drive is configured to read the first data file from one or more of the one or more first data storage devices faster than the second drive is configured to read the first data file from one or more of the one or more second data storage devices; and
wherein the first computing device comprises one or more hardware processors, and further comprises the first drive and the second drive, and wherein the first computing device is configured with computer-executable instructions that, when executed, cause the first computing device to:
receive, from the client computing device, the first data file in the native format,
store the first data file in the native format in the first drive,
after a first amount of time, take a file-level snapshot of primary data that is stored in the first drive,
based on the file-level snapshot, determine that the first data file has changed since a preceding snapshot of primary data in the first drive,
move the first data file in the native format from the first drive to the second drive,
replace the first data file on the first drive with a stub that points to the first data file on the second drive, and
after a second amount of time longer than the first amount of time, (a) convert the first data file from the native format into one or more secondary copies that are in a secondary copy format, wherein the secondary copy format is distinct from the native format, and (b) store the one or more secondary copies, in the secondary copy format, in the second drive, and
responsive to a request for at least part of the first data file, depending on a timing of the request compared to the first amount of time and the second amount of time, serve at least part of the first data file from one of: the first drive and the second drive.
12. The system of claim 11 , wherein the computer-executable instructions, when executed, further cause the first computing device to: after a third amount of time longer than the second amount of time, store the one or more secondary copies, in the secondary copy format, at a secondary storage device that is distinct from and in communication with the first computing device; and
wherein, responsive to the request received after the one or more secondary copies are stored at the secondary storage device, (A) restore the one or more secondary copies from the secondary copy format into the first data file in the native format at one of: the first drive and the second drive, and (B) transmit at least part of the first data file as restored in the native format.
13. The system of claim 12 , wherein the computer-executable instructions, when executed, further cause the first computing device to: update the stub to point to the one or more secondary copies at the secondary storage device.
14. The system of claim 11 , wherein the computer-executable instructions, when executed, further cause the first computing device to:
for storage at one or more of the first drive and the second drive, divide the first data file received from the client computing device into a plurality of file extents that each have a smaller size than the first data file, and
responsive to the request, serve a first file extent among the plurality of file extents.
15. The system of claim 11 , wherein the computer-executable instructions, when executed, further cause the first computing device to: based on the request being received before the first data file has been moved to the second drive, transmit at least part of the first data file in the native format from the first drive.
16. The system of claim 11 , wherein the computer-executable instructions, when executed, further cause the first computing device to: based on the request being received after the first data file has been moved to the second drive and before the first data file has been converted into the one or more secondary copies, transmit at least part of the first data file in the native format from the second drive.
17. The system of claim 11 , wherein the computer-executable instructions, when executed, further cause the first computing device to: based on the request being received after the first data file has been converted into the one or more secondary copies stored at the second drive, (A) restore the one or more secondary copies from the secondary copy format into the first data file in the native format, and (B) transmit at least part of the first data file in the native format from the second drive.
18. The system of claim 11 , wherein the first drive is configured in the system as a faster storage tier than a first portion of the second drive that stores the first data file in the native format, and wherein a second portion of the second drive that stores the one or more secondary copies in the secondary copy format is configured in the system as a slower storage tier than the first portion of the second drive.
19. The system of claim 11 , wherein a storage manager that executes on a third computing device is configured to manage one or more storage policies that define the first amount of time and the second amount of time, wherein the third computing device is distinct from one or more of: the client computing device and the first computing device, and wherein the third computing device comprises one or more hardware processors.
20. The system of claim 11 , wherein the first drive is configured to execute a first file system and wherein the second drive is configured to execute a second file system, which is distinct from, and of a different type than, the first file system.