Forwarding data amongst cooperative computing devices of a massive data ingestion system

1. A method for ingesting a large amount of data in a storage network, wherein the method is executed by one or more computing devices of a plurality of computing devices of the storage network, wherein the method comprises:

receiving, by a first group of computing devices of the plurality of computing devices, data objects having a first data type identifier of a plurality of data type identifiers, wherein the plurality of data type identifiers correspond to a plurality of data types associated with the data objects;

interpreting, by the first group of computing devices, the data objects having the first data type identifier to sort, based on sorting criteria, the data objects having the first data type identifier into a first processing category and a second processing category;

error encoding, by the first group of computing devices, the data objects in the second processing category based on short term storage error encoding parameters to produce pluralities of sets of encoded data slices; and

sending, by the first group of computing devices, the pluralities of sets of encoded data slices to a set of storage and execution units for storage therein.

2. The method of claim 1 , wherein the plurality of data types comprises two or more of:

a data file;

an email;

a blog;

a white paper;

a social media post; and

graphics.

3. The method of claim 1 , wherein the sorting criteria comprises one or more of:

a word search;

an image search;

destination information;

facial recognition;

an encryption type; and

source information.

4. The method of claim 1 further comprises:

receiving, by a second group of computing devices of the plurality of computing devices, data objects having a second data type identifier of the plurality of data type identifiers;

interpreting, by the second group of computing devices, the data objects having the second data type identifier to sort, based on second sorting criteria, the data objects having the second data type identifier into a third processing category and a fourth processing category;

error encoding, by the second group of computing devices, the data objects in the fourth processing category based on short term storage error encoding parameters to produce second pluralities of sets of encoded data slices; and

sending, by the second group of computing devices, the second pluralities of sets of encoded data slices to a second set of storage and execution units for storage.

5. The method of claim 1 further comprises:

discarding, by the first group of computing devices, the data objects of the first processing category.

6. The method of claim 1 further comprises:

sending, by the first group of computing devices, the data objects of the first processing category to a second set of storage and execution units for storage.

7. The method of claim 1 further comprises:

interpreting, by the set of storage and execution units, the pluralities of sets of encoded data slices to determine data objects of interest of the data objects of the second processing category based on interest criteria;

error encoding, by the set of storage and execution units, the data objects of interest based on long term storage error encoding parameters to produce second pluralities of sets of encoded data slices; and

sending, by the set of storage and execution units, the second pluralities of sets of encoded data slices to a second set of storage and execution unit units for storage and subsequent processing.

8. The method of claim 1 further comprises:

obtaining, by the one or more computing devices, the data objects;

as the data objects are obtained:

assigning a data type identifier of the plurality of data type identifiers to each data object of the data objects based on the data object being a data type of the plurality of data types;

sending the data objects having the first data type identifier to the first group of computing devices; and

sending data objects having a second data type identifier to a second group of computing devices of the plurality of computing devices.

9. A computer readable memory device of a storage network comprises:

at least one memory element for storing operational instructions that, when executed by a first group of computing devices of a plurality of computing devices of the storage network, causes the first group of computing devices to:

receive data objects having a first data type identifier of a plurality of data type identifiers, wherein the plurality of data type identifiers correspond to a plurality of data types associated with the data objects;

interpret the data objects having the first data type identifier to sort, based on sorting criteria, the data objects having the first data type identifier into a first processing category and a second processing category;

error encode the data objects in the second processing category based on short term storage error encoding parameters to produce pluralities of sets of encoded data slices; and

send the pluralities of sets of encoded data slices to a set of storage and execution units for storage.

10. The computer readable memory device of claim 9 , wherein the plurality of data types comprises two or more of:

a data file;

an email;

a blog;

a white paper;

a social media post; and

graphics.

11. The computer readable memory device of claim 9 , wherein the sorting criteria comprises one or more of:

a word search;

an image search;

destination information;

facial recognition;

an encryption type; and

source information.

12. The computer readable memory device of claim 9 , wherein the at least one memory element stores further operational instructions that, when executed by a second group of computing devices of the plurality of computing devices, causes the second group of computing devices to:

interpret the data objects having a second data type identifier of the plurality of data type identifiers to sort, based on second sorting criteria, the data objects having the second data type identifier into a third processing category and a fourth processing category;

dispersed storage error encode the data objects in the fourth processing category based on short term storage error encoding parameters to produce second pluralities of sets of encoded data slices; and

send the second pluralities of sets of encoded data slices to a second set of storage and execution units for storage.

13. The computer readable memory device of claim 9 , wherein the at least one memory element stores further operational instructions that, when executed by the first group of computing devices, causes the first group of computing devices to:

discard the data objects of the first processing category.

14. The computer readable memory device of claim 9 , wherein the at least one memory element stores further operational instructions that, when executed by the first group of computing devices, causes the first group of computing devices to:

send the data objects of the first processing category to a second set of storage and execution units for storage.

15. The computer readable memory device of claim 9 , where the at least one memory element for stores further operational instructions that, when executed by the set of storage and execution units, causes the set of storage and execution units to:

interpret the pluralities of sets of encoded data slices to determine data objects of interest of the data objects of the second processing category based on interest criteria;

dispersed storage error encode the data objects of interest based on long term storage error encoding parameters to produce second pluralities of sets of encoded data slices; and

send the second pluralities of sets of encoded data slices to a second set of storage and execution unit units for storage and subsequent processing.

16. The computer readable memory device of claim 9 , wherein the at least one memory element stores further operational instructions that, when executed by one or more computing devices of the plurality of computing devices, causes the one or more computing devices to:

obtain the data objects; and

as the data objects are obtained:

assign a data type identifier of the plurality of data type identifiers to each data object of the data objects based on the data object being a data type of the plurality of data types;

send the data objects having the first data type identifier to the first group of computing devices; and

send data objects having a second data type identifier to a second group of computing devices of the plurality of computing devices.