Method and apparatus for dispersed storage of streaming data
View Patent ↗A method begins by receiving streaming data. The method continues by partitioning the streaming data into a first data stream and a second data stream. For the first data stream, the method continues by encoding, in accordance with error coding dispersed storage function parameters, the first data stream into a first encoded data slices. The method continues by determining first memory of a DSN to store the first encoded data slices and facilitating storage of the first encoded data slices in the first memory. For the second data stream, the method continues by encoding, in accordance with the error coding dispersed storage function parameters, the second data stream into a second encoded data slices. The method continues by determining second memory of the DSN to store the second encoded data slices and facilitating storage of the second encoded data slices in the second memory.
1. A method for execution by a processing module of a computing device, the method comprises:
receiving streaming data;
partitioning the streaming data into a first data stream and a second data stream;
for the first data stream:
encoding, in accordance with error coding dispersed storage function parameters, the first data stream into a first plurality of sets of encoded data slices;
determining first memory of a dispersed storage network (DSN) to store the first plurality of sets of encoded data slices; and
facilitating storage of the first plurality of sets of encoded data slices in the first memory; and
for the second data stream:
encoding, in accordance with the error coding dispersed storage function parameters, the second data stream into a second plurality of sets of encoded data slices;
determining second memory of the DSN to store the second plurality of sets of encoded data slices; and
facilitating storage of the second plurality of sets of encoded data slices in the second memory.
2. The method of claim 1 further comprises:
selecting a first set of dispersed storage resources for the first data stream; and
selecting a second set of dispersed storage resources of the second data stream.
3. The method of claim 1 further comprises:
selecting a first set of storage nodes of the DSN for the first memory; and
selecting a second set of storage nodes of the DSN for the second memory, wherein the first and second sets of storage nodes are mutually exclusive.
4. The method of claim 1 further comprises:
selecting a first set of storage nodes of the DSN for the first memory; and
selecting a second set of storage nodes of the DSN for the second memory, wherein one or more storage nodes is common to the first and second sets of storage nodes.
5. The method of claim 1 further comprises:
receiving metadata with the streaming data; and
determining the first and second memories based on the metadata.
6. The method of claim 1 further comprises:
receiving metadata with the streaming data; and
determining to partition the stream of data into the first and second data streams based on the metadata.
7. The method of claim 1 , wherein the partitioning the streaming data into a first data stream and a second data stream further comprises:
iteratively dividing the streaming data into groups, wherein each group includes three portions; and
for a group of the groups:
using a first of the three portions for the first data stream;
using a second of the three portions for the second data stream; and
discarding a third of the three portions.
8. The method of claim 1 further comprises:
encoding, in accordance with a first set of the error coding dispersed storage function parameters, the first data stream into the first plurality of sets of encoded data slices; and
encoding, in accordance with a second set of the error coding dispersed storage function parameters, the second data stream into the second plurality of sets of encoded data slices.
9. A dispersed storage computing device comprises:
an interface;
a processing module;
a first set of dispersed storage resources; and
a second set of dispersed storage resources, wherein:
the interface receives streaming data;
the processing module partitions the streaming data into a first data stream and a second data stream;
the first set of dispersed storage resources encodes, in accordance with error coding dispersed storage function parameters, the first data stream into a first plurality of sets of encoded data slices;
the processing module determines first memory of a dispersed storage network (DSN) to store the first plurality of sets of encoded data slices; and
the interface outputs the first plurality of sets of encoded data slices for storage in the first memory;
the second set of dispersed storage resources encodes, in accordance with the error coding dispersed storage function parameters, the second data stream into a second plurality of sets of encoded data slices;
the processing module determines second memory of the DSN to store the second plurality of sets of encoded data slices; and
the interface outputs the second plurality of sets of encoded data slices for storage in the second memory.
10. The dispersed storage computing device of claim 9 , wherein the processing module is further operable to:
select the first set of dispersed storage resources from a plurality of dispersed storage resources; and
select the second set of dispersed storage resources from the plurality of dispersed storage resources.
11. The dispersed storage computing device of claim 9 , wherein the processing module is further operable to:
select a first set of storage nodes of the DSN for the first memory; and
select a second set of storage nodes of the DSN for the second memory, wherein the first and second sets of storage nodes are mutually exclusive.
12. The dispersed storage computing device of claim 9 , wherein the processing module is further operable to:
select a first set of storage nodes of the DSN for the first memory; and
select a second set of storage nodes of the DSN for the second memory, wherein one or more storage nodes is common to the first and second sets of storage nodes.
13. The dispersed storage computing device of claim 9 , wherein the processing module is further operable to:
receive, via the interface, metadata with the streaming data; and
determine the first and second memories based on the metadata.
14. The dispersed storage computing device of claim 9 , wherein the processing module is further operable to:
receive, via the interface, metadata with the streaming data; and
determine to partition the stream of data into the first and second data streams based on the metadata.
15. The dispersed storage computing device of claim 9 , wherein the processing module is further operable to partition the streaming data into a first data stream and a second data stream by:
iteratively dividing the streaming data into groups, wherein each group includes three portions; and
for a group of the groups:
using a first of the three portions for the first data stream;
using a second of the three portions for the second data stream; and
discarding a third of the three portions.
16. The dispersed storage computing device of claim 9 :
the first set of dispersed storage resources encodes, in accordance with a first set of the error coding dispersed storage function parameters, the first data stream into the first plurality of sets of encoded data slices; and
the second set of dispersed storage resources encodes, in accordance with a second set of the error coding dispersed storage function parameters, the second data stream into the second plurality of sets of encoded data slices.