Creating a new file for a dispersed storage network
A method begins by a dispersed storage (DS) processing module receiving a file creation request to create a new file from portions of a set of existing files. The method continues with the DS processing module identifying data segments associated with the set of existing files corresponding to the portions of the set of existing files to produce identified data segments and identifying sets of encoded data slices of the identified data segments to produce identified sets of encoded data slices. The method continues with the DS processing module determining addressing information for the identified sets of encoded data slices from existing addressing information of the set of existing files and generating a segment allocation table for the new file based on the addressing information of the identified sets of encoded data slices such that the new file is created without duplication of the portions of the existing files.
1. A method for execution by a computing device, the method comprises:
receiving a file creation request to create a new data file from portions of a set of existing files containing data, wherein an existing file of the set of existing files includes a plurality of data segments that is stored as a plurality of sets of encoded data slices in a dispersed storage network (DSN) memory;
identifying, in accordance with the file creation request, data segments from the pluralities of data segments of the set of existing files corresponding to the portions of the set of existing files to produce identified data segments;
identifying, based on the identified data segments, sets of encoded data slices of the pluralities of sets of encoded data slices of the identified data segments to produce identified sets of encoded data slices;
determining addressing information for the identified sets of encoded data slices from existing addressing information of the set of existing files; and
creating the new data file by generating a segment allocation table for the new data file based on the addressing information of the identified sets of encoded data slices such that the new data file is created without duplication of the portions of the existing files containing the data.
2. The method of claim 1 further comprises:
the existing file of the set of existing files further includes a set of data regions, wherein a data region of the set of data regions includes a consecutive subset of the plurality of data segments;
identifying, in accordance with the file creation request, data regions from the set of data regions of the set of existing files corresponding to the portions of the set of existing files to produce identified data regions;
identifying, based on the identified data regions, the sets of encoded data slices of the plurality of sets of encoded data slices of the identified data regions to produce the identified sets of encoded data slices; and
determining the addressing information for the identified sets of encoded data slices from a set of segment allocation tables associated with the set of existing files, wherein the set of segment allocation tables includes the existing addressing information of the set of existing files at data region level.
3. The method of claim 2 , wherein the identifying the sets of encoded data slices comprises:
retrieving a set of segment allocation tables for the set of existing files based on the file creation request, wherein a segment allocation table of the set of segment allocation tables includes a plurality of entries, and wherein an entry of the plurality of entries includes a DSN address regarding storage of a data region of the set of data segments of the set of existing files in the DSN memory and segmentation information regarding the dividing of the data region into the consecutive subset of the plurality of data segments.
4. The method of claim 2 , wherein addressing information for an identified set of encoded data slices of the identified sets of encoded data slices comprises:
a start segment vault source name corresponding to a data region associated with the identified set of encoded data slices; and
segmentation information including one or more of:
a segment size of data segments of the data region;
a total length of the data region; and
a segmentation approach of the data region.
5. The method of claim 1 , wherein the generating the segment allocation table for the new data file comprises:
identifying consecutive data segments of the identified data segments;
grouping the consecutive data segments into a region; and
generating addressing information for the region, which is stored as an entry in the segment allocation table, wherein individual addressing information of the identified sets of encoded data slices is determinable from the addressing information of the region.
6. The method of claim 1 , wherein the identifying data segments comprises at least one of:
receiving, within the file creation request, identification of the identified data segments; and
searching, based on a search criteria, the set of existing files, or addressing information thereof, to identify the data segments.
7. The method of claim 1 , wherein the identifying the sets of encoded data slices comprises:
retrieving a set of segment allocation tables for the set of existing files based on the file creation request, wherein a segment allocation table of the set of segment allocation tables includes a plurality of entries, and wherein an entry of the plurality of entries includes a DSN address regarding storage of a data segment of the plurality of data segments in the DSN memory.
8. The method of claim 1 further comprises:
obtaining a segment allocation table vault source name for the segment allocation table;
encoding the segment allocation table using a dispersed storage error coding function to produce encoded table slices; and
outputting the encoded table slices to the DSN memory for storage therein utilizing the segment allocation table vault source name.
9. A dispersed storage (DS) computing device comprises:
an interface;
memory; and
a processing circuit operably coupled to the interface and to the memory, wherein the processing circuit is operable to:
receive a file creation request to create a new data file from portions of a set of existing files containing data, wherein an existing file of the set of existing files includes a plurality of data segments that is stored as a plurality of sets of encoded data slices in a dispersed storage network (DSN) memory;
identify, in accordance with the file creation request, data segments from the plurality of data segments of the set of existing files corresponding to the portions of the set of existing files to produce identified data segments;
identify, based on the identified data segments, sets of encoded data slices of the plurality of sets of encoded data slices of the identified data segments to produce identified sets of encoded data slices;
determine addressing information for the identified sets of encoded data slices from existing addressing information of the set of existing files; and
create the new data file by generating a segment allocation table for the new data file based on the addressing information of the identified sets of encoded data slices such that the new data file is created without duplication of the portions of the existing files containing the data.
10. The DS computing device of claim 9 further comprises:
the existing file of the set of existing files further includes a set of data regions, wherein a data region of the set of data regions includes a consecutive subset of the plurality of data segments;
the processing circuit further functions to:
identify, in accordance with the file creation request, data regions from the set of data regions of the set of existing files corresponding to the portions of the set of existing files to produce identified data regions;
identify, based on the identified data regions, the sets of encoded data slices of the plurality of sets of encoded data slices of the identified data regions to produce the identified sets of encoded data slices; and
determine the addressing information for the identified sets of encoded data slices from a set of segment allocation tables associated with the set of existing files, wherein the set of segment allocation tables includes the existing addressing information of the set of existing files at data region level.
11. The DS computing device of claim 10 , wherein the processing circuit functions to identify the sets of encoded data slices by:
retrieving a set of segment allocation tables for the set of existing files based on the file creation request, wherein a segment allocation table of the set of segment allocation tables includes a plurality of entries, and wherein an entry of the plurality of entries includes a DSN address regarding storage of a data region of the set of data segments of the set of existing files in the DSN memory and segmentation information regarding the dividing of the data region into the consecutive subset of the plurality of data segments.
12. The DS computing device of claim 10 , wherein addressing information for an identified set of encoded data slices of the identified sets of encoded data slices comprises:
a start segment vault source name corresponding to a data region associated with the identified set of encoded data slices; and
segmentation information including one or more of:
a segment size of data segments of the data region;
a total length of the data region; and
a segmentation approach of the data region.
13. The DS computing device of claim 9 , wherein the processing circuit functions to generate the segment allocation table for the new data file by:
identifying consecutive data segments of the identified data segments;
grouping the consecutive data segments into a region; and
generating addressing information for the region, which is stored as an entry in the segment allocation table, wherein individual addressing information of the identified sets of encoded data slices is determinable from the addressing information of the region.
14. The DS computing device of claim 9 , wherein the processing circuit functions to identify data segments by at least one of:
receiving, within the file creation request, identification of the identified data segments; and
searching, based on a search criteria, the set of existing files, or addressing information thereof, to identify the data segments.
15. The DS computing device of claim 9 , wherein the processing circuit functions to identify the sets of encoded data slices by:
retrieving a set of segment allocation tables for the set of existing files based on the file creation request, wherein a segment allocation table of the set of segment allocation tables includes a plurality of entries, and wherein an entry of the plurality of entries includes a DSN address regarding storage of a data segment of the plurality of data segments in the DSN memory.
16. The DS computing device of claim 9 further comprises:
the processing circuit further functions to:
obtain a segment allocation table vault source name for the segment allocation table;
encode the segment allocation table using a dispersed storage error coding function to produce encoded table slices; and
output the encoded table slices to the DSN memory for storage therein utilizing the segment allocation table vault source name.