Event prediction
A system for event prediction is configured to ingest at least partially incomplete unstructured data from at least one data source, convert at least a subset of the ingested data into a prediction model in accordance with a received model definition, receive a prediction query, and in response, supply an event prediction based on the prediction model. In some embodiments, ingesting the data involves transforming the unstructured data into a universal structured format in which the data is represented as key-value pairs each comprising a key tuple and an associated value, the key tuple specifying an operation and at least one identifier; the prediction query includes a key tuple specifying an operation and at least one identifier; and the event prediction generated with the prediction model includes a value associated with the key tuple that is missing from the transformed data.
1. A system for event prediction, the system comprising:
a hardware processor; and
a memory storing non-transitory instructions, executed by the hardware processor, for:
ingesting at least partially incomplete unstructured data from at least one data source, the ingesting comprising transforming the unstructured data into a universal structured format that allows for structured predictive query processing, the ingested data being represented in the universal structured format as key-value pairs each comprising a key tuple and an associated value, the key tuple specifying an operation and at least one identifier;
receiving a model definition and converting at least a subset of the ingested data into a prediction model in accordance with the received definition;
incrementally updating the prediction model in response to incremental changes in the ingested data;
receiving a prediction query comprising a key tuple specifying an operation and at least one identifier; and
using the prediction model to generate and output an event prediction in response to the received prediction query, wherein the generated event prediction comprises a value associated with the key tuple that is missing from the transformed data.
2. The system of claim 1 , wherein at least one of the key tuples specifies two identifiers.
3. The system of claim 1 , further comprising a data store for storing the transformed data, the data store in communication with the at least one data source via an associated data connector.
4. The system of claim 1 , wherein the at least one data source comprises at least one of a flat file, a database, or a source of streaming data.
5. The system of claim 1 , wherein the system is configured to continuously ingest new data from the at least one data source.
6. The system of claim 1 , wherein the prediction query is a SQL query related to the at least partially incomplete unstructured data.
7. The system of claim 1 , wherein the ingested data includes at least one of a numeric identifier, written text, location data, or imagery.
8. The system of claim 1 , wherein the prediction query is a time-series query related to data gathered over a time period and the event prediction includes a predicted future value.
9. The system of claim 1 , wherein the prediction query is a classification-based query related to an entity and the event prediction includes a classification of the entity.
10. The system of claim 1 , wherein the prediction query is related to a new entity, and the event prediction includes a predicted attribute about the new entity.
11. A method for event prediction using a computer comprising a processor and a memory containing non-transitory instructions configured to be executed by the processor, the method comprising:
ingesting at least partially incomplete unstructured data from at least one data source, the ingesting comprising transforming the unstructured data into a universal structured format that allows for structured predictive query processing, the ingested data being represented in the universal structured format as key-value pairs each comprising a key tuple and an associated value, the key tuple specifying an operation and at least one identifier;
receiving a model definition and converting at least a subset of the ingested data into a prediction model in accordance with the received definition;
incrementally updating the prediction model in response to incremental changes in the ingested data;
receiving a prediction query comprising a key tuple specifying an operation and at least one identifier; and
using the prediction model to generate and output an event prediction in response to the received prediction query, wherein the generated event prediction comprises a value associated with the key tuple that is missing from the transformed data.
12. The method of claim 11 , wherein at least one of the key tuples specifies two identifiers.
13. The method of claim 11 , wherein the at least one data source comprises at least one of a flat file, a database, or a source of streaming data.
14. The method of claim 11 , wherein new data is continuously ingested from the at least one data source.
15. The method of claim 11 , wherein the prediction query is a SQL query related to the at least partially incomplete unstructured data.
16. The method of claim 11 , wherein the ingested data includes at least one of a numeric identifier, written text, location data, or imagery.
17. The method of claim 11 , wherein the prediction query is a time-series query related to data gathered over a time period and the event prediction includes a predicted future value.
18. The method of claim 11 , wherein the prediction query is a classification-based query related to an entity and the event prediction includes a classification of the entity.
19. The method of claim 11 , wherein the prediction query is related to a new entity, and the event prediction includes a predicted attribute about the new entity.
20. A non-transitory computer readable storage medium containing instructions configured to be executed by a processor to cause the processor to perform operations comprising:
ingesting at least partially incomplete unstructured data from at least one data source, the ingesting comprising transforming the unstructured data into a universal structured format that allows for structured predictive query processing, the ingested data being represented in the universal structured format as key-value pairs each comprising a key tuple and an associated value, the key tuple specifying an operation and at least one identifier;
receiving a model definition and converting at least a subset of the ingested data into a prediction model in accordance with the received definition;
incrementally updating the prediction model in response to incremental changes in the ingested data;
receiving a prediction query comprising a key tuple specifying an operation and at least one identifier; and
using the prediction model to generate and output an event prediction in response to the received prediction query, wherein the generated event prediction comprises a value associated with the key tuple that is missing from the transformed data.