Generating and morphing a collection of files in a folder/sub-folder structure that collectively has desired dedupability, compression, clustering and commonality
One example method includes receiving a set of filesystem parameters, creating a simulated filesystem based on the filesystem parameters, receiving a set of target characteristics for a file collection, based on the target characteristics, slicing a datastream into a grouping of data slices, populating the simulated files with the data slices to create the file collection and forward or reverse morphing the file collection from one generation to another without rewriting the entire file collection.
1. A method, comprising the operations:
receiving a generation-X (gen-X) datastream and a generation-N (gen-N) datastream, where X=N+α, α is any integer greater than or equal to 1, and, N and X are respective integers;
comparing part of the gen-X datastream with part of the gen-N datastream;
when a portion of a file included in a file collection has different respective generation values in the gen-X datastream and the gen-N datastream, and each of the respective generation values corresponding to a different version of the portion of the file, either:
overwriting the portion of the file in the file collection with the portion of the file in the gen-N datastream; or
overwriting the portion of the file in the file collection with the portion of the file in the gen-X datastream; and
repeating the comparing and overwriting processes until either:
the file collection has been morphed into a gen-N file collection in which, for each portion of the gen-X datastream whose generation value in the gen-X datastream was different from its generation value in the gen-N datastream, that portion of the gen-X datastream has been overwritten, in the gen-N file collection, with a portion of the gen-N datastream corresponding to the generation value in the gen-N datastream or:
the file collection has been morphed into a gen-X file collection in which, for each portion of the gen-N datastream whose generation value in the gen-N datastream was different from its generation value in the gen-X datastream, that portion of the gen-N datastream has been overwritten, in the gen-X file collection, with a portion of the gen-X datastream corresponding to the value in the gen-X datastream.
2. The method as recited in claim 1 , wherein the gen-N datastream predates the gen-X datastream.
3. The method as recited in claim 1 , wherein the gen-X and gen-N datastreams are compared sequentially.
4. The method as recited in claim 1 , wherein only modified portions of file, rather than entire files, are overwritten.
5. The method as recited in claim 1 , wherein each of the gen-X and gen-N datastreams possesses target characteristics that comprise one or more of dedupability, compressibility, commonality, and clustering.
6. The method as recited in claim 1 , wherein the gen-X datastream is more than one generation removed from the gen-N datastream.
7. The method as recited in claim 1 , wherein each of the gen-X and gen-N datastreams possesses target characteristics that comprise one or more of compressibility, commonality, and clustering, and the target characteristics are collectively present in the gen-X file collection when the file collection has been morphed into the gen-X file collection , and in the gen-N file collection when the file collection has been morphed into the gen-N file collection.
8. The method as recited in claim 1 , wherein the operations further comprise backing up with a backup application, and deduplicating, the gen-X file collection when the file collection has been morphed into the gen-X file collection, or backing up with the backup application, and deduplicating, the gen-N file collection when the file collection has been morphed into the gen-N file collection.
9. A non-transitory storage medium having stored therein instructions which are executable by one or more hardware processors to perform operations comprising:
receiving a generation-X (gen-X) datastream and a generation-N (gen-N) datastream, where X=N+α, α is any integer greater than or equal to 1, and, N and X are respective integers;
comparing part of the gen-X datastream with part of the gen-N datastream;
when a portion of a file included in a file collection has different respective generation values in the gen-X datastream and the gen-N datastream, and each of the respective generation values corresponding to a different version of the portion of the file, either:
overwriting the portion of the file in the file collection with the portion of the file in the gen-N datastream; or
overwriting the portion of the file in the file collection with the portion of the file in the gen-X datastream; and
repeating the comparing and overwriting processes until either:
the file collection has been morphed into a gen-N file collection in which, for each portion of the gen-X datastream whose generation value in the gen-X datastream was different from its generation value in the gen-N datastream, that portion of the gen-X datastream has been overwritten, in the gen-N file collection, with a portion of the gen-N datastream corresponding to the generation value in the gen-N datastream or:
the file collection has been morphed into a gen-X file collection in which, for each portion of the gen-N datastream whose generation value in the gen-N datastream was different from its generation value in the gen-X datastream, that portion of the gen-N datastream has been overwritten, in the gen-X file collection, with a portion of the gen-X datastream corresponding to the value in the gen-X datastream.
10. The non-transitory storage medium as recited in claim 9 , wherein the gen-N datastream predates the gen-X datastream.
11. The non-transitory storage medium as recited in claim 9 , wherein the gen-X and gen-N datastreams are compared sequentially.
12. The non-transitory storage medium as recited in claim 9 , wherein only modified portions of file, rather than entire files, are overwritten.
13. The non-transitory storage medium as recited in claim 9 , wherein each of the gen-X and gen-N datastreams possesses target characteristics that comprise one or more of dedupability, compressibility, commonality, and clustering.
14. The non-transitory storage medium as recited in claim 9 , wherein the gen-X datastream is more than one generation removed from the gen-N datastream.
15. The non-transitory storage medium as recited in claim 9 , wherein each of the gen-X and gen-N datastreams possesses target characteristics that comprise one or more of compressibility, commonality, and clustering, and the target characteristics are collectively present in the gen-X file collection when the file collection has been morphed into the gen-X file collection, and in the gen-N file collection when the file collection has been morphed into the gen-N file collection.
16. The non-transitory storage medium as recited in claim 9 , wherein the operations further comprise backing up with a backup application, and deduplicating, the gen-X file collection when the file collection has been morphed into the gen-X file collection, or backing up with the backup application, and deduplicating, the gen-N file collection when the file collection has been morphed into the gen-N file collection.