Data distillery for signal detection
Computer-implemented methods, systems and products for analytics and discovery of patterns or signals. The method includes a set of operations or steps, including collecting data from a plurality of data sources, the data having a plurality of associated data types, and filtering the collected data based on identifying viable data sources from which the data is collected. The method further includes prioritizing discovery objectives based on analyzing the filtering results, and enriching the filtered collected data from viable data sources according to the prioritized discovery objectives. The method further includes extracting one or more signals from the enriched data using one or more machine learning mechanisms in combination with qualified subject matter expertise input, and graphically displaying the extracted signals in a meaningful way to a human operator such that the human operator is enabled to understand importance of extracted signals.
1. A system comprising:
at least one programmable processor;
a non-transitory machine-readable medium storing instructions that, when executed by the at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
receiving information framing a discovery objective regarding aspects of a plan;
receiving information identified by one or more users, the information identifying data sources deemed by the one or more users to be relevant to the discovery objective;
identifying other data sources based on at least one of the received information identifying data sources deemed by the one or more users to be relevant to the discovery objective and the received information framing the discovery objective;
retrieving data based on the information framing the discovery objective from at least some of the data sources deemed to be relevant to the discovery objective by the one or more users and the identified other data sources;
assessing quality of individual data sources from which data was retrieved and calculating quality indicators indicative of the assessed quality of the individual data sources;
providing, over a network to one or more participants, the calculated quality indicators indicative of the assessed quality of the data sources from which data was retrieved;
receiving information indicative of one or more data sources for which the quality indicators are calculated;
receiving one or more datasets of the data retrieved from at least some of the data sources deemed to be relevant to the discovery objective by the one or more users or from the identified other data sources; and
wrangling the one or more datasets into a form that is computationally actionable by a user.
2. The system of claim 1 , wherein the operations further comprise:
extracting one or more signals from the one or more datasets, using one or more machine learning mechanisms in combination with qualified subject matter expertise input; and
graphically displaying the extracted signals to a human operator.
3. The system of claim 1 , wherein the operations further comprise:
enriching at least some data from the one or more datasets to generate an enriched form of at least some data corresponding to the one or more datasets, the enriched form being computationally actionable by a user.
4. The system of claim 3 , wherein the operations further comprise:
extracting one or more signals from the enriched data using one or more machine learning mechanisms in combination with qualified subject matter expertise input;
wherein enriching at least some data from the one or more datasets comprises combining one or more data elements from the one or more datasets to create characteristics and variables that make the one or more extracted signals more explicit.
5. The system of claim 3 , wherein the operations further comprise:
processing the one or more datasets and the enriched form of at least some data to identify one or more of relationships, anomalies and patterns within the one or more datasets.
6. A computer program product comprising a non-transitory machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
receiving information framing a discovery objective regarding aspects of a plan;
receiving information identified by one or more users, the received information identifying data sources deemed by the one or more users to be relevant to the discovery objective;
identifying other data sources based on at least one of the received information identifying data sources deemed by the one or more users to be relevant to the discovery objective and the received information framing the discovery objective;
retrieving data based on the information framing the discovery objective from at least some of the data sources deemed to be relevant to the discovery objective by the one or more users and the identified other data sources;
assessing quality of individual data sources from which data was retrieved and calculating quality indicators indicative of the assessed quality of the individual data sources;
providing, over a network to one or more participants, the calculated quality indicators indicative of the assessed quality of the data sources from which data was retrieved;
receiving information indicative of one or more data sources for which the quality indicators are calculated; and
receiving one or more datasets of the data retrieved from at least some of the data sources deemed to be relevant to the discovery objective by the one or more users or from the identified other data sources; and
wrangling the one or more datasets into a form that is computationally actionable by a user.
7. The computer program product of claim 6 , wherein the operations further comprise:
extracting one or more signals from the one or more datasets, using one or more machine learning mechanisms in combination with qualified subject matter expertise input; and
graphically displaying the extracted signals to a human operator.
8. The computer program product of claim 6 , wherein the operations further comprise:
enriching at least some data from the one or more datasets to generate an enriched form of at least some data corresponding to the one or more datasets, the enriched form being computationally actionable by a user.
9. The computer program product of claim 8 , wherein the operations further comprise:
extracting one or more signals from the enriched data using one or more machine learning mechanisms in combination with qualified subject matter expertise input; and
wherein enriching at least some data from the one or more datasets comprises combining one or more data elements from the one or more datasets to create characteristics and variables that make the one or more signals more explicit.
10. The computer program product of claim 8 , wherein the operations further comprise:
processing the one or more datasets and the enriched form of at least some data to identify one or more of relationships, anomalies and patterns within the one or more datasets.
11. A computer-implemented method executable by one or more processors, the method comprising:
receiving information framing a discovery objective regarding aspects of a plan;
receiving information identified by one or more users, the information identifying data sources deemed by the one or more users to be relevant to the discovery objective;
identifying other data sources based on at least one of the received information identifying data sources deemed by the one or more users to be relevant to the discovery objective and the received information framing the discovery objective;
retrieving data based on the information framing the discovery objective from at least some of the data sources deemed to be relevant to the discovery objective by the one or more users and the identified other data sources;
assessing quality of individual data sources from which data was retrieved and calculating quality indicators indicative of the assessed quality of the individual data sources;
providing, over a network to one or more participants, the calculated quality indicators indicative of the assessed quality of the data sources from which data was retrieved;
receiving information indicative of one or more data sources for which the quality indicators are calculated;
receiving one or more datasets of the data retrieved from at least some of the data sources deemed to be relevant to the discovery objective by the one or more users or from the identified other data sources; and
wrangling the one or more datasets into a form that is computationally actionable by a user.
12. The method of claim 11 further comprising:
extracting one or more signals from the one or more datasets, using one or more machine learning mechanisms in combination with qualified subject matter expertise input; and
graphically displaying the extracted signals to a human operator.
13. The method of claim 11 further comprising:
enriching at least some data from the one or more datasets to generate an enriched form of at least some data corresponding to the one or more datasets, the enriched form being computationally actionable by a user.
14. The method of claim 13 further comprising:
extracting one or more signals from the enriched data using one or more machine learning mechanisms in combination with qualified subject matter expertise input,
wherein enriching at least some data from the one or more datasets comprises combining one or more data elements from the one or more datasets to create characteristics and variables that make the one or more extracted signals more explicit.
15. The method of claim 13 further comprising:
processing the one or more datasets and the enriched form of at least some data to identify one or more of relationships, anomalies and patterns within the one or more datasets.