Systems, methods, and apparatus for subdividing data for storage in a dispersed data storage grid
View Patent ↗An efficient method for breaking source data into smaller data subsets and storing those subsets along with coded information about some of the other data subsets on different storage nodes such that the original data can be recreated from a portion of those data subsets in an efficient manner.
1. A computer comprises:
a coding module operably coupled to:
segment a computer file into one or more strings of data;
for each of the one or more strings of data:
create a plurality of subsets of data;
encode, using a coding algorithm, the plurality of subsets of data to create a plurality of coded values; and
create a plurality of data slices from the plurality of subsets of data and the plurality of coded values; and
a communication network interface operably coupled to transmit the plurality of data slices of each of the one or more strings of data.
2. The computer of claim 1 , wherein the coding algorithm further comprises:
a failure tolerance level that indicates a threshold of slices of the plurality of data slices that are needed to reconstruct the corresponding string of data.
3. The computer of claim 1 , wherein the communication network interface further functions to:
transmit the plurality of data slices to a receiving computer.
4. The computer of claim 1 , wherein the communication network interface further functions to:
transmit the plurality of data slices to a plurality of storage nodes.
5. The computer of claim 4 , wherein the communication network interface further functions to:
transmit a first data slice of the plurality of data slices of each of the one or more strings of data to a first storage node of the plurality of storage nodes;
transmit a second data slice of the plurality of data slices of each of the one or more strings of data to a second storage node of the plurality of storage nodes;
transmit a third data slice of the plurality of data slices of each of the one or more strings of data to a third storage node of the plurality of storage nodes; and
transmit a fourth data slice of the plurality of data slices of each of the one or more strings of data to a fourth storage node of the plurality of storage nodes.
6. The computer of claim 1 , wherein the coding algorithm further comprises:
a convolutional encoding algorithm.
7. The computer of claim 1 , further comprises:
a coding module operably coupled to:
segment a file system into a plurality of strings of data, wherein the file system includes the computer file;
for each of the plurality of strings of data:
create the plurality of subsets of data;
encode, using the coding algorithm, the plurality of subsets of data to create the plurality of coded values; and
create the plurality of data slices from the plurality of subsets of data and the plurality of coded values.
8. The computer of claim 1 , further comprises:
the computer file is encrypted.
9. A method for execution by a computer, the method comprises:
segmenting a computer file into one or more strings of data;
for each of the one or more strings of data:
creating a plurality of subsets of data;
encoding, using a coding algorithm, the plurality of subsets of data to create a plurality of coded values; and
creating a plurality of data slices from the plurality of subsets of data and the plurality of coded values; and
transmitting the plurality of data slices of each of the one or more strings of data.
10. The method of claim 9 , wherein the coding algorithm further comprises:
a failure tolerance level that indicates a threshold of slices of the plurality of data slices that are needed to reconstruct the corresponding string of data.
11. The method of claim 9 , wherein the transmitting further comprises:
transmitting the plurality of data slices to a receiving computer.
12. The method of claim 9 , wherein the transmitting further comprises:
transmitting the plurality of data slices to a plurality of storage nodes.
13. The method of claim 12 , wherein the transmitting further comprises:
transmitting a first data slice of the plurality of data slices of each of the one or more strings of data to a first storage node of the plurality of storage nodes;
transmitting a second data slice of the plurality of data slices of each of the one or more strings of data to a second storage node of the plurality of storage nodes;
transmitting a third data slice of the plurality of data slices of each of the one or more strings of data to a third storage node of the plurality of storage nodes; and
transmitting a fourth data slice of the plurality of data slices of each of the one or more strings of data to a fourth storage node of the plurality of storage nodes.
14. The method of claim 9 , wherein the coding algorithm further comprises:
a convolutional encoding algorithm.
15. The method of claim 14 , further comprises:
segmenting a file system into a plurality of strings of data, wherein the file system includes the computer file;
for each of the plurality of strings of data:
creating the plurality of subsets of data;
encoding, using the coding algorithm, the plurality of subsets of data to create the plurality of coded values; and
creating the plurality of data slices from the plurality of subsets of data and the plurality of coded values.
16. The method of claim 9 , further comprises:
the computer file is encrypted.
17. The method of claim 16 , further comprises:
decrypting the computer file.