Efficient storage of data in a dispersed storage network
View Patent ↗A method begins by a dispersed storage (DS) processing module receiving data for storage and generating a dispersed storage network (DSN) source name for the data. The method continues with the DS processing module determining whether substantially identical data to the data has been previously stored in memory of the DSN. When the substantially identical data has been previously stored in the memory of the DSN, the method continues with the DS processing module generating an object linking file that links the data to the substantially identical data, dispersed storage error encoding the object linking file to produce a set of encoded link file slices, and outputting the set of encoded link file slices for storage in the memory of the DSN.
1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:
receiving a data object “B” for storage;
generating a DSN source name for the data object “B”;
when the data object “B” is substantially identical to a previously stored data object “A”, generating an object linking file that links a DSN address of the previously stored data object “A” to the data object “B”, wherein the previously stored data object “A” has an associated DSN source name that differs from the DSN source name for the data object “B”;
dispersed storage error encoding the object linking file to produce a set of encoded link file slices; and
outputting the set of encoded link file slices for storage in memory of the DSN.
2. The method of claim 1 , wherein the generating the DSN source name for the data object “B” comprises:
obtaining a vault identifier;
generating a random number; and
combining the vault identifier and the random number to produce the DSN source name.
3. The method of claim 1 further comprises:
the data object “B” is substantially identical to the previously stored data object “A” when a data tag of data object “B” substantially matches a data tag of data object “A”.
4. The method of claim 1 , further comprises:
the data object “B” is substantially identical to the previously stored data object “A” when a data comparison function performed on the data object “B” and the previously stored data object “A” indicates that the data object “B” is substantially identical the previously stored data object “A”.
5. The method of claim 1 , wherein the generating the object linking file further comprises:
labeling the object linking file with the DSN source name for the data object “B”.
6. The method of claim 5 , wherein the generating the object linking file further comprises:
generating a data tag;
accessing, based on the data tag, a data tag list to identify an entry associated with the previously stored data object “A”; and
retrieving the DSN address of the previously stored data object “A” from the identified entry.
7. The method of claim 5 , wherein the generating the object linking file further comprises:
generating a data tag;
searching, based on the data tag, a dispersed hierarchical index to identify an entry associated with the previously stored data object “A”; and
retrieving the DSN address of the previously stored data object “A” from the identified entry.
8. A dispersed storage (DS) module comprises:
a first module, when operable within a computing device, causes the computing device to:
receive a data object “B” for storage; and
generate a dispersed storage network (DSN) source name for the data object “B”;
a second module, when operable within the computing device, causes the computing device to:
when the data object “B” is substantially identical to a previously stored data object “A” generate an object linking file that links a DSN address of the previously stored data object “A” to the data object “B”;
dispersed storage error encode the object linking file to produce a set of encoded link file slices; and
output the set of encoded link file slices for storage in memory of the DSN.
9. The DS module of claim 8 , wherein the first module functions to cause the computing device to generate the DSN source name for the data object “B” by:
obtaining a vault identifier;
generating a random number; and
combining the vault identifier and the random number to produce the DSN source name.
10. The DS module of claim 8 further comprises:
the data object “B” is substantially identical to the previously stored data object “A” when a data tag of data object “B” substantially matches a data tag of data object “A”.
11. The DS module of claim 8 further comprises:
the data object “B” is substantially identical to the previously stored data object “A” when a data comparison function performed on the data object “B” and the previously stored data object “A” indicates that the data object “B” is substantially identical the previously stored data object “A”.
12. The DS module of claim 8 , wherein the second module functions to cause the computing device to generate the object linking file by:
labeling the object linking file with the DSN source name for the data object “B”.
13. The DS module of claim 12 , wherein the second module further functions to cause the computing device to generate the object linking file by:
generating a data tag;
accessing, based on the data tag, a data tag list to identify an entry associated with the previously stored data object “A”; and
retrieving the DSN address of the previously stored data object “A” from the identified entry.
14. The DS module of claim 12 , wherein the second module further functions to cause the computing device to generate the object linking file by:
generating a data tag;
searching, based on the data tag, a dispersed hierarchical index to identify an entry associated with the previously stored data object “A”; and
retrieving the DSN address of the previously stored data object “A” from the identified entry.