Distributed data analytics
An apparatus in one embodiment comprises a distributed data processing system in which multiple processing devices communicate with one another over at least one network. The distributed data processing system is configured to obtain reads of biological samples of respective sample sources, with each of the biological samples containing genomic material from a plurality of distinct microorganisms within an environment of a corresponding one of the sample sources, and to perform distributed data analytics to characterize an actual or potential outbreak of at least one of a disease, an infection and a contamination that involves genomic material from multiple ones of the distinct microorganisms in one or more of the sample sources. Performing distributed data analytics illustratively comprises performing local analytics in respective ones of a plurality of data zones, and performing global analytics utilizing results of the local analytics performed in the respective data zones.
1. A method comprising:
obtaining reads of biological samples of respective sample sources wherein each of the biological samples contains genomic material from a plurality of distinct microorganisms within an environment of a corresponding one of the sample sources; and
performing distributed data analytics to characterize an actual or potential outbreak of at least one of a disease, an infection and a contamination that involves genomic material from multiple ones of the distinct microorganisms in one or more of the sample sources;
wherein performing distributed data analytics comprises:
performing local analytics in respective ones of a plurality of data zones; and
performing global analytics utilizing results of the local analytics performed in the respective data zones;
wherein each of the data zones comprises one or more sequencing centers utilized to generate a corresponding subset of the reads within that data zone;
wherein the local analytics performed in a given one of the data zones utilize reads of one or more of the biological samples sequenced in the one or more sequencing centers of the given data zone;
wherein the local analytics performed in the given data zone comprise analyzing the reads of the one or more biological samples against a local set of known gene units;
wherein at least one result of the global analytics comprises a graph in which nodes correspond to respective biological samples and edges between the nodes characterize epidemiological relationships between the biological samples, the edges being weighted by sample-to-sample comparison scores of metagenomics sequencing results for the biological samples; and
wherein the method is implemented by a distributed data processing system comprising a plurality of processing devices configured to communicate with one another over at least one network.
2. The method of claim 1 wherein the local set of known gene units for the given data zone comprises locally-available pre-processed sequenced genomic data collected from reads of the one or more biological samples sequenced in the one or more sequencing centers of the given data zone.
3. The method of claim 2 wherein at least one known gene unit in the local set of known gene units for the given data zone is augmented with metadata comprising one or more of patient symptoms, time and location.
4. The method of claim 1 wherein the reads of the biological samples comprise respective sets of gene units sequenced from those biological samples in corresponding ones of a plurality of sequencing centers.
5. The method of claim 1 wherein the data zones comprise respective geographically-distributed regional data centers each configured to perform local analytics utilizing locally accessible data resources of its corresponding data zone and wherein the results of the local analytics performed in the respective data zones preserve for each of the data zones at least one specified policy of that data zone relating to at least one of privacy, security, governance, risk and compliance.
6. The method of claim 1 wherein at least one additional result of the global analytics comprises global view information characterizing relationships between a first set of reads of a first set of one or more biological samples sequenced in a first set of one or more sequencing centers of a first one of the plurality of data zones and one or more additional sets of reads of one or more additional sets of one or more biological samples sequenced in one or more additional sets of one or more sequencing centers of one or more additional ones of the plurality of data zones.
7. The method of claim 6 wherein the global view information comprises one or more portions of the graph in which the nodes correspond to respective biological samples sequenced by respective ones of the first and one or more additional sets of sequencing centers and the edges between the nodes are weighted by the sample-to-sample comparison scores of the metagenomics sequencing results for the first and one or more additional sets of one or more biological samples.
8. The method of claim 6 wherein the global view information comprises a global epidemiological graph computed utilizing a plurality of local epidemiological graphs provided as results of the local analytics performed in respective ones of the plurality of data zones.
9. The method of claim 6 further comprising utilizing the global view information to at least one of:
predict a spread pattern for the actual or potential outbreak of the at least one disease, infection and contamination; and
identify failures in one or more preventive or sanitary controls.
10. The method of claim 6 wherein performing the global analytics utilizing the results of the local analytics performed in the respective data zones comprises applying a community contact detection algorithm in conjunction with generating the global view information.
11. A method comprising:
obtaining reads of biological samples of respective sample sources wherein each of the biological samples contains genomic material from a plurality of distinct microorganisms within an environment of a corresponding one of the sample sources; and
performing distributed data analytics to characterize an actual or potential outbreak of at least one of a disease, an infection and a contamination that involves genomic material from multiple ones of the distinct microorganisms in one or more of the sample sources;
wherein performing distributed data analytics comprises:
performing local analytics in respective ones of a plurality of data zones; and
performing global analytics utilizing results of the local analytics performed in the respective data zones;
wherein each of the data zones comprises one or more sequencing centers utilized to generate a corresponding subset of the reads within that data zone;
wherein the local analytics performed in a given one of the data zones utilize reads of one or more of the biological samples sequenced in the one or more sequencing centers of the given data zone;
wherein the local analytics performed in the given data zone comprise analyzing the reads of the one or more biological samples against a local set of known gene units;
wherein at least one result of the global analytics comprises at least one of a genomic comparison component and an epidemiologic comparison component characterizing relationships between a first set of reads of a first set of one or more biological samples sequenced in a first set of one or more sequencing centers of a first one of the plurality of data zones and one or more additional sets of reads of one or more additional sets of one or more biological samples sequenced in one or more additional sets of one or more sequencing centers of one or more additional ones of the plurality of data zones; and
wherein the method is implemented by a distributed data processing system comprising a plurality of processing devices configured to communicate with one another over at least one network.
12. The method of claim 11 wherein the genomic comparison component comprises hit abundance score vectors for respective ones of a plurality of biological samples that are identified as being related to the at least one disease, infection and contamination based at least in part on the first and one or more additional sets of biological samples sequenced in the first and one or more additional sets of one or more sequencing centers, wherein the hit abundance score vector for a given one of the plurality of biological samples comprises a plurality of entries corresponding to respective occurrence frequencies of at least one read of the given biological sample in respective target gene units.
13. The method of claim 11 wherein the epidemiologic comparison component comprises a patient graph in which nodes corresponding to patients are connected in the patient graph based at least in part on patient comparative indexes.
14. The method of claim 11 wherein at least one additional result of the global analytics further comprises a combination of portions of the genomic comparison component with portions of the epidemiologic comparison component that characterizes the actual or potential outbreak of the at least one disease, infection and contamination.
15. The method of claim 11 wherein performing the global analytics further comprises generating an outbreak graph for the at least one disease, infection and contamination utilizing both the genomic comparison component and the epidemiologic comparison component.
16. The method of claim 15 wherein performing the global analytics further comprises applying a community contact detection algorithm as a preprocessing operation prior to generating the outbreak graph.
17. The method of claim 11 wherein performing the global analytics further comprises applying a biclustering operation to a hit abundance score matrix comprising a plurality of hit abundance score vectors based at least in part on epidemiological data associated with different ones of a plurality of patients.
18. The method of claim 11 further comprising utilizing one or more results of the global analytics to associate a particular patient with the at least one disease, infection and contamination based at least in part on statistical decision making utilizing respective portions of the genomic comparison component and the epidemiologic comparison component.
19. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by a distributed data processing system comprising a plurality of processing devices configured to communicate with one another over at least one network causes said distributed data processing system:
to obtain reads of biological samples of respective sample sources wherein each of the biological samples contains genomic material from a plurality of distinct microorganisms within an environment of a corresponding one of the sample sources; and
to perform distributed data analytics to characterize an actual or potential outbreak of at least one of a disease, an infection and a contamination that involves genomic material from multiple ones of the distinct microorganisms in one or more of the sample sources;
wherein performing distributed data analytics comprises:
performing local analytics in respective ones of a plurality of data zones; and
performing global analytics utilizing results of the local analytics performed in the respective data zones;
wherein each of the data zones comprises one or more sequencing centers utilized to generate a corresponding subset of the reads within that data zone;
wherein the local analytics performed in a given one of the data zones utilize reads of one or more of the biological samples sequenced in the one or more sequencing centers of the given data zone;
wherein the local analytics performed in the given data zone comprise analyzing the reads of the one or more biological samples against a local set of known gene units; and
wherein at least one result of the global analytics comprises a graph in which nodes correspond to respective biological samples and edges between the nodes characterize epidemiological relationships between the biological samples, the edges being weighted by sample-to-sample comparison scores of metagenomics sequencing results for the biological samples.
20. An apparatus comprising:
a distributed data processing system comprising a plurality of processing devices configured to communicate with one another over at least one network;
wherein said distributed data processing system is configured:
to obtain reads of biological samples of respective sample sources wherein each of the biological samples contains genomic material from a plurality of distinct microorganisms within an environment of a corresponding one of the sample sources; and
to perform distributed data analytics to characterize an actual or potential outbreak of at least one of a disease, an infection and a contamination that involves genomic material from multiple ones of the distinct microorganisms in one or more of the sample sources;
wherein performing distributed data analytics comprises:
performing local analytics in respective ones of a plurality of data zones; and
performing global analytics utilizing results of the local analytics performed in the respective data zones;
wherein each of the data zones comprises one or more sequencing centers utilized to generate a corresponding subset of the reads within that data zone;
wherein the local analytics performed in a given one of the data zones utilize reads of one or more of the biological samples sequenced in the one or more sequencing centers of the given data zone;
wherein the local analytics performed in the given data zone comprise analyzing the reads of the one or more biological samples against a local set of known gene units; and
wherein at least one result of the global analytics comprises a graph in which nodes correspond to respective biological samples and edges between the nodes characterize epidemiological relationships between the biological samples, the edges being weighted by sample-to-sample comparison scores of metagenomics sequencing results for the biological samples.