System and method for model-based prediction using a distributed computational graph workflow
A system and method for crowd-sourced refinement of natural phenomenon for risk management and contract validation, comprising at least a heterogeneous mixture of sensors and data-gathering techniques, a sensor fusion suite, and a business operating system, which ingests, transforms if necessary, and analyzes received data and develops and applies models of prediction of consequences of the sensor data and future events based on such data for purposes such as insurance liability and risk assessment, emergency services planning, and financial market predictions, and comparing historical models and data with current data and models to attempt to refine and utilize a more precise predictive model for these purposes.
1. A system for model-based prediction using a distributed computational graph workflow, comprising:
a computing device comprising a memory, a processor, and a non-volatile data storage device; at least one physical sensor;
a webcrawler comprising a first plurality of programming instructions stored in the memory and operating on the processor, wherein the first plurality of programming instructions, when operating on the processor, causes the computing device to:
automatically gather data from a plurality of Internet-enabled sources including social media, search engine results, website data, and news outlets; send gathered data to a sensor fusion suite over a network;
a distributed computational graph module comprising a second plurality of programming instructions stored in the memory and operating on the processor, wherein the second plurality of programming instructions, when operating on the processor, causes the computing device to:
receive a first model for processing from the sensor fusion suite;
construct a distributed computational graph from the first model, the distributed computational graph representing a data processing workflow and comprising a directed graph with nodes representing data transformations and edges representing messaging between the nodes;
receive input data from the sensor fusion suite;
and processing the input data according to the data processing workflow of the distributed computational graph;
and send the processed data to the sensor fusion suite;
and the sensor fusion suite comprising a third plurality of programming instructions stored in the memory and operating on the processor, wherein the third plurality of programming instructions, when operating on the processor, causes the computing device to:
receive input data comprising gathered data from the webcrawler and at the physical sensor over a network;
record the input data and the time of receipt in a multidimensional time-series database stored on the non-volatile data storage device;
retrieve a second model from the non-volatile data storage device for prediction of a future event from the received data;
send the received data to the distributed computational graph module for processing;
receive the processed data from the distributed computational graph module;
predict a future event from the processed data;
and make the results of the prediction available for viewing to an operating system.
2. The system of claim 1 , wherein the sensor fusion suite further receives input data from a network-enabled financial market crawler, which may automatically gather data from financial markets including stock prices and trades.
3. The system of claim 1 , wherein the sensor fusion suite further receives input data from manual data entry, including human observers during a natural disaster entering data into the sensor fusion suite over a network.
4. The system of claim 1 , wherein the sensor fusion suite further receives input data from network connected cameras, including either or all of video, audio, and image data.
5. The system of claim 1 , wherein the sensor fusion suite uses machine learning algorithms to make the prediction.
6. The system of claim 5 , wherein the machine learning techniques include gamification of data through adversarial training of either or both of current and historic predictive models.
7. A method for model-based prediction using a distributed computational graph workflow, comprising the steps of: using a webcrawler operating on a computing device comprising a memory, a processor, and a non-volatile data storage device, automatically gather data from a plurality of Internet-enabled sources including social media, search engine results, website data, and news outlets;
receive data from a physical sensor;
using a distributed computational graph module operating on the computing device:
receive a first model for processing from the sensor fusion suite;
construct a distributed computational graph from the first model, the distributed computational graph representing a data processing workflow and comprising a directed graph with nodes representing data transformations and edges representing messaging between the nodes;
receive input data from the sensor fusion suite;
and processing the input data according to the data processing workflow of the distributed computational graph;
and send the processed data to the sensor fusion suite;
and using the sensor fusion suite operating on the computing device:
receive input data comprising gathered data from the webcrawler and the physical sensor over a network;
record the input data and the time of reception in a multidimensional time-series database stored on the non-volatile data storage device;
retrieve a second model from the non-volatile data storage device for prediction of a future event from the received data;
send the received data to the distributed computational graph module for processing;
receive the processed data from the distributed computational graph module;
predict a future event from the processed data;
and make the results of the prediction available for viewing to an operating system.
8. The method of claim 7 , wherein the sensor fusion suite further receives input data from a network-enabled financial market crawler, which may automatically gather data from financial markets including stock prices and trades.
9. The method of claim 7 , wherein the sensor fusion suite further receives input data from manual data entry, including human observers during a natural disaster entering data into the sensor fusion suite over a network.
10. The method of claim 7 , wherein the sensor fusion suite further receives input data from network connected cameras, including either or all of video, audio, and image data.
11. The method of claim 7 , wherein the sensor fusion suite uses machine learning algorithms to make the prediction.
12. The method of claim 11 , wherein the machine learning techniques include gamification of data through adversarial training of either or both of current and historic predictive models.