System and methods for implementing a server-based hierarchical mass storage system
Setting up and supporting the computer infrastructure for a remote satellite office is a difficult task for any information technology department. To simplify the task, an integrated server system with a hierarchical storage system is proposed. The hierarchical storage system includes the ability to store data at an off-site cloud storage service. The server system is remotely configurable and thus allows the server to be configured and populated with data from a remote location.
1. A method of implementing a hierarchical data storage within a computer system, said method comprising:
installing a virtual storage device driver on the computer system;
creating a hierarchical storage stack on said computer system for storing data, said hierarchical storage stack comprising:
a local data storage layer coupled with local data storage that is local to said computer system, and
a remote data storage layer coupled with a remote data storage service;
determining that available storage capacity of the local data storage layer exceeds a capacity size limitation;
identifying, among the data objects stored by the local data storage layer, selected data objects that are accessed at a lower access frequency than other data objects stored by the hierarchical storage stack; and
relocating the selected data objects from the local data storage layer to the remote data storage layer, wherein the relocating raises the available storage capacity of the local data storage layer above the capacity limit.
2. The method of implementing a hierarchical data storage system as set forth in claim 1 wherein said virtual storage device driver comprises a Storport driver executing in a Windows Server.
3. The method of implementing a hierarchical data storage system as set forth in claim 1 wherein said virtual storage device driver comprises a virtual SCSI driver executing in a Linux Server.
4. The method of implementing a hierarchical data storage system as set forth in claim 1 , said method further comprising:
executing an application program, said application program directly accessing said virtual storage device driver.
5. The method of implementing a hierarchical data storage system as set forth in claim 1 wherein said computer system comprises a cloud-based virtual server.
6. The method of implementing a hierarchical data storage system as set forth in claim 5 wherein said hierarchical data storage system exposes a network storage interface for applications to access.
7. The method of implementing a hierarchical data storage system as set forth in claim 1 , said method further comprising:
receiving a request to create a new data volume in said administration layer of said hierarchical data storage system;
creating said new data volume in said hierarchical storage stack that will initially store data in said local data storage layer and later spill data to said remote data storage layer that stores data with a remote data storage service.
8. The method of implementing a hierarchical data storage system as set forth in claim 1 wherein said at least one local data storage layer comprises a first local data storage layer that stores data in a solid-state memory system and a second local data storage layer that stores data on a hard disk drive.
9. The method of implementing a hierarchical data storage system as set forth in claim 1 , said method further comprising:
receiving a request to create a back-up of a data volume stored in said hierarchical data storage system;
creating a snapshot volume of said data volume in said hierarchical data storage system; and
copying data from said snapshot volume stored in said at least one local data storage layer to said remote data storage service.
10. A data server of a hierarchical data storage system within a computer system, comprising:
a processor; and
a memory storing instructions that, when executed by the processor, provide:
a hierarchical storage stack on said computer system, said hierarchical storage stack comprising:
a local data storage layer coupled with local data storage that is local to said computer system, and
a remote data storage layer coupled with a remote data storage service; and
a virtual device driver that:
determines that available storage capacity of the local data storage layer exceeds a capacity size limitation;
identifies, among the data objects stored by the local data storage layer, selected data objects that are accessed at a lower access frequency than other data objects stored by the hierarchical storage stack; and
relocates the selected data objects from the local data storage layer to the remote data storage layer, wherein the relocating raises the available storage capacity of the local data storage layer above the capacity limit.
11. The data server as set forth in claim 10 wherein said virtual storage device driver comprises a Storport driver executing in a Windows Server.
12. The data server as set forth in claim 10 wherein said computer system comprises a cloud-based virtual server.
13. The data server as set forth in claim 10 wherein said at least one local data storage layer comprises a first local data storage layer that stores data in a raw format and a second local data storage layer that stores data in a format with duplicate data removed.
14. The data server as set forth in claim 10 wherein said at least one local data storage layer comprises a first local data storage layer that stores data in a solid-state memory system and a second local data storage layer that stores data on a hard disk drive.
15. The data server as set forth in claim 10 wherein said set of computer instructions further perform the operations of:
receiving a request to create a back-up of a data volume stored in said hierarchical data storage system;
creating a snapshot volume of said data volume in said hierarchical data storage system; and
copying data from said snapshot volume stored in said at least one local data storage layer to said remote data storage service.
16. A hierarchical data storage system for a computer system, comprising:
a hierarchical storage stack on said computer system, said hierarchical storage stack comprising:
a local data storage layer coupled with a local storage device, and
a remote data storage layer coupled with a remote data storage service; and
a virtual storage device driver that:
determines that available storage capacity of the local data storage layer exceeds a capacity size limitation;
identifies, among the data objects stored by the local data storage layer, selected data objects that are accessed at a lower access frequency than other data objects stored by the hierarchical storage stack; and
relocates the selected data objects from the local data storage layer to the remote data storage layer, wherein the relocating raises the available storage capacity of the local data storage layer above the capacity limit.
17. The hierarchical data storage system of claim 16 , wherein said hierarchical data storage system exposes a network storage interface for applications to access.
18. The hierarchical data storage system of claim 16 , further comprising:
a data volume in said hierarchical storage stack that will initially store data in said local data storage layer and later spill data to said remote data storage layer that stores data with a remote data storage service.
19. The method of claim 1 , further comprising: utilizing the local data storage layer as a read-ahead cache for the remote data storage layer by:
identifying a selected data object of the remote data storage layer is absent in the local data storage layer; and
before receiving an incoming request to access the selected data object, moving the selected data object from the remote data storage layer to the local data storage layer.