Providing access to state information associated with operators in a data processing system

1 . A data processing system comprising:

one or more processors;

an operator that is executable by the one or more processors to generate an output from input data received from data sources, wherein the operator comprises state information based on the input data that is not provided as output, and wherein the state information is not queried from the operator;

a configuration manager for obtaining the state information from the operator and providing a copy of the state information to an object interface;

the object interface that provides access to an object graph that has a node that symbolically represents the operator by storing a copy of the state information of the operator, wherein the object graph is a separate data structure from the operator; and

a query manager that is executable by the one or more processors to process a graph query to retrieve the copy of the state information from the node of the object graph via the object interface, wherein the query manager performs an operation using the copy of the state information.

2 . The data processing system of claim 1 , wherein:

the node comprises a property that stores the copy of the state information of the operator;

retrieving the copy of the state information from the node comprises retrieving the copy of the state information from the property of the node, and

the state information cannot be queried from the operator.

3 . The data processing system of claim 1 , further comprising a set of operators that include the operator, and wherein:

the set of operators internally stores multiple versions of state information corresponding to input data having different timestamps; and

the multiple versions of state information are inaccessible by querying the set of operators.

4 . The data processing system of claim 3 , wherein:

the graph query specifies a temporal constraint; and

the multiple versions of the state information correspond to a temporal dimension.

5 . The data processing system of claim 3 , wherein:

the object graph is an object graph that comprises a set of nodes that correspond to the set of operators; and

copies of the multiple versions of state information of the set of operators are accessible via the set of nodes that correspond to the set of operators.

6 . The data processing system of claim 5 , further comprising:

utilizing a first set of operators at a first point in time to process inputs;

dynamically changing the first set of operators to a second set of operators at a second point in time to process the inputs, wherein the first set of operators and the second set of operators comprise the operator, and wherein the first set of operators differs from the second set of operators; and

in response to dynamically changing the first set of operators to the second set of operators, updating the set of nodes in the object graph to correspond to the second set of operators.

7 . The data processing system of claim 1 , wherein the graph query comprises a reference to a previous point in time, and the data processing system further comprises:

a data store;

system checkpoints stored in the data store;

events stored in the data store; and

a state information manager that is executable by the one or more processors to determine state information corresponding to the previous point in time using a system checkpoint and subsequent events that have a timestamp after the system checkpoint and not later than the previous point in time specified by the graph query.

8 . The data processing system of claim 1 , further comprising a configuration manager that configures the operator to internally store the state information for a time period that exceeds a defined failover time period.

9 . The data processing system of claim 1 , wherein the object interface is an object graph interface that provides indirect access to the state information via graph queries processed in relation to the object graph.

10 . The data processing system of claim 1 , wherein:

the operator performs a computation with respect to the input data, and

the object graph comprises a representation of the computation.

11 . A system for providing access to state information within operators of a data processing system, comprising:

a processor; and

a computer memory comprising instructions that, when executed by the processor, cause the system to perform operations comprising:

generating an output within an operator of the data processing system based on input data received from a data source, wherein the operator comprises state information based on the input data that is not provided as output, and wherein the state information is not queried from the operator;

obtaining, by a configuration manager, the state information from the operator and providing a copy of the state information to an object interface;

accessing, via the object interface, an object graph that has a node that symbolically represents the operator by storing a copy of the state information of the operator, wherein the object graph is a separate data structure from the operator; and

processing a graph query to retrieve the copy of the state information from the node of the object graph via the object interface, wherein processing a graph query includes performing an operation using the copy of the state information.

12 . The system of claim 11 , wherein:

the node comprises a property that stores the copy of the state information of the operator; and

retrieving the copy of the state information from the node comprises retrieving the copy of the state information from the property of the node.

13 . The system of claim 11 , further comprising a set of operators that include the operator, and wherein:

the set of operators internally stores multiple versions of state information corresponding to input data having different timestamps; and

the multiple versions of state information are inaccessible by querying the set of operators.

14 . The system of claim 13 , wherein:

the graph query specifies a temporal constraint; and

the multiple versions of the state information correspond to a temporal dimension.

15 . The system of claim 13 , wherein:

the object graph is an object graph that comprises a set of nodes that correspond to the set of operators; and

copies of the multiple versions of state information of the set of operators are accessible via the set of nodes that correspond to the set of operators.

16 . The system of claim 15 , further comprising:

utilizing a first set of operators at a first point in time to process inputs;

dynamically changing the first set of operators to a second set of operators at a second point in time to process the inputs, wherein the first set of operators and the second set of operators comprise the operator, and wherein the first set of operators differs from the second set of operators; and

in response to dynamically changing the first set of operators to the second set of operators, updating the set of nodes in the object graph to correspond to the second set of operators.

17 . The system of claim 11 , wherein the graph query comprises a reference to a previous point in time, and the data processing system further comprises:

a data store;

system checkpoints stored in the data store;

events stored in the data store; and

a state information manager that is executable by the processor to determine state information corresponding to the previous point in time using a system checkpoint and subsequent events that have a timestamp after the system checkpoint and not later than the previous point in time specified by the graph query.

18 . The system of claim 11 , further comprising a configuration manager that configures the operator to internally store the state information for a time period that exceeds a defined failover time period.

19 . A method for providing access to state information within operators of a data processing system, comprising:

generating an output within an operator of the data processing system based on input data received from a data source, wherein the operator comprises state information based on the input data that is not provided as output, and wherein the state information is not queried from the operator;

obtaining, by a configuration manager, the state information from the operator and providing a copy of the state information to an object interface;

accessing, via the object interface, an object graph that has a node that symbolically represents the operator by storing a copy of the state information of the operator, wherein the object graph is a separate data structure from the operator; and

processing a graph query to retrieve the copy of the state information from the node of the object graph via the object interface, wherein processing a graph query includes performing an operation using the copy of the state information.

20 . The method of claim 19 , wherein the state information is hidden state information that is not provided as output from the operator.