THIN PROVISIONED CLONE
A unified system provides primary storage and in-line analytics-based data protection. Additional data intelligence and analytics gathered on protected data and prior analytics are stored in discovery points. The disclosed system implements multi-threaded log writes across rimary and restore nodes with write gathering across file systems; nested directories such as may be used for storing virtual machine files, where every subdirectory has an associated file system for snapshot purposes; and cloning objects on demand with background metadata and data migration.
1 . A method for processing in a data-intelligent storage system that includes a primary node storing primary data and in-line analytics, a restore node storing further analytics and intelligence, comprising the steps of:
intercepting a request to clone a data object;
thin provisioning a clone object;
opening the clone object for access; and
copying data to the clone object only upon the first to occur of:
(a) a subsequent access request for the clone object; and
(b) as part of a background restore process.
2 . The method of claim 1 wherein the step of thin provisioning the clone object further comprises:
creating a bitmap data object containing a bit for each one of a plurality data chunks in the data object.
3 . The method of claim 2 further comprising:
setting bits in the bitmap corresponding to data chunks referred to in the subsequent access request for the clone.
4 . The method of claim 2 wherein the background restore process further comprises:
setting bits in the bitmap corresponding to data chunks as they are copied to the clone object.
5 . The method of claim 1 wherein the data object is a directory object containing one or more files arranged in at least one directory, and the step of thin provisioning further comprises:
creating a tree structure in the clone object that include metadata for the at least one directory.
6 . The method of claim 5 wherein the background restore process additionally comprises:
initiating a depth-first search of the directory object to determine subdirectory structures; and
storing metadata concerning the subdirectory structures in the clone object.
7 . The method of claim 6 wherein the background restore process further comprises, after initiating a depth-first search,
initiating a breadth-first search of the directory object.
8 . The method of claim 7 wherein the step of initiating a breadth-first search is performed as a concurrent thread with a thread executing the depth-first search.
9 . The method of claim 2 wherein the clone object is a temporary clone object, and further comprising:
when the subsequent access request to the clone is a read access, reading the data object; and
when the subsequent access request to the clone is a write access, writing data to the clone.
10 . The method of claim 9 additionally comprising:
creating a bitmap data object containing a bit for each one of a plurality data chunks in the data object; and
when the subsequent access request to the clone is a read access, further:
reading bits in the bitmap corresponding to requested data chunks, and
when such bits are not set, reading the data object to satisfy the access request.
11 . The method of claim 1 wherein the data-intelligent storage system comprises a protection and analytics (PART) node, and the method further comprises:
handling access requests to the clone at the PART node in multiple concurrently executing threads;
assigning a transaction identifier to the access requests; and
logging each access request with its associated transaction identifier in a random access, multithreaded log.
12 . The method of claim 11 additionally comprising:
forwarding access requests to the primary and restore nodes;
logging access requests in a single threaded sequential log in each of the primary and restore nodes; and
storing data associated with the access request in a cache memory in each of the primary and restore nodes.