Providing access to state information associated with operators in a data processing system
A data processing system that provides access to operator state information includes a plurality of operators that are configured to perform a computation with respect to data received from data sources. State information is associated with at least one of the plurality of operators. The data processing system also includes an object graph that comprises a representation of the computation, and that may dynamically change at runtime. The data processing system also includes an interface that provides access to the state information via the object graph. The data processing system also includes a query manager that is executable to process a graph query to retrieve the state information by traversing a plurality of nodes within the object graph. Temporal navigation is also supported. Thus, processing a graph query may involve navigating to a node in the object graph at a certain point in time.
1. A method for providing access to hidden state information within operators of a data processing system, comprising:
generating state information within an operator of the data processing system based on an input received from a data source;
generating an output utilizing the operator based on the state information, wherein the state information is not included in the output;
receiving, from a querying entity, a graph query to retrieve the state information from the operator, wherein the state information is not available via querying the operator;
based on the graph query, accessing a node of an object information data structure via an object interface, wherein the node symbolically represents the operator by storing a copy of the state information of the operator, and wherein the object information data structure is a separate data structure from the operator that provides access to a node; and
providing the copy of the state information from the object information data structure in response to the querying entity.
2. The method of claim 1 , wherein:
the object information data structure is an object graph that comprises multiple nodes that correspond to multiple operators;
each of the multiple nodes comprises one or more properties that comprise copies of hidden state information of the multiple operators;
the hidden state information with the multiple operators is not accessible by direct queries; and
the copies of the hidden state information of the multiple operators are accessible via the object interface.
3. The method of claim 1 , further comprising:
navigating to the node within the object information data structure;
selecting a property of the node; and
creating an event processing query with respect to the property of the node.
4. The method of claim 1 , wherein:
the operator internally stores the state information for a plurality of different points in time;
the graph query specifies a temporal constraint; and
the state information comprises a temporal dimension.
5. The method of claim 1 , further comprising:
receiving user input that specifies a time period for storing the state information in the operator; and
modifying the operator to maintain the state information for a duration of the time period.
6. A method for accessing hidden state information from operators within a data processing system, comprising:
generating state information within an operator of the data processing system based on an input received from a data source;
generating an output utilizing the operator based on the state information, wherein the state information is not outputted from the operator;
providing, to the data processing system, a graph query comprising a request to determine the state information of the operator;
based on the graph query, accessing a node of an object information data structure via an object interface, wherein the node symbolically represents the operator by storing a copy of the state information of the operator, and wherein the object information data structure is a separate data structure from the operator that provides access to the node; and
receiving, from the data processing system, the copy of the state information obtained from the node in the object information data structure.
7. The method of claim 6 , wherein accessing the node comprises traversing a set of nodes in the object information data structure to identify the node of the set of nodes that corresponds to the operator, wherein the set of nodes in the object information data structure symbolically represents a corresponding set of operators in the data processing system.
8. The method of claim 6 , further comprising accessing the copy of the state information from the node by identifying a property of the node comprising the copy of the state information.
9. The method of claim 6 , wherein:
the data processing system comprises an event processing system;
the data source comprises an event source; and
the operator processes one or more events from the event source based on timestamps of the one or more events.
10. The method of claim 6 , wherein the object information data structure is an object graph that dynamically updates the copy of the state information in the node as changes to the state information in the operator are detected.
11. A system for providing access to hidden state information within operators of a data processing system, comprising:
a processor; and
a computer memory comprising instructions that, when executed by the processor, cause the system to perform operations comprising:
generating state information within an operator of the data processing system based on an input received from a data source;
generating an output utilizing the operator based on the state information, wherein the state information is not included in the output;
receiving, from a querying entity, a graph query to retrieve the state information from the operator, wherein the state information is not available via querying the operator;
based on the graph query, accessing a node of an object information data structure via an object interface, wherein the node symbolically represents the operator by storing a copy of the state information of the operator, and wherein the object information data structure is a separate data structure from the operator that provides access to a node; and
providing the copy of the state information from the object information data structure in response to the querying entity.
12. The system of claim 11 , wherein:
the object information data structure is an object graph that comprises multiple nodes that correspond to multiple operators;
each of the multiple nodes comprises one or more properties that comprise copies of hidden state information of the multiple operators;
the hidden state information with the multiple operators is not accessible by direct queries; and
the copies of the hidden state information of the multiple operators are accessible via the object interface.
13. The system of claim 11 , wherein the operations further comprise:
navigating to the node within the object information data structure;
selecting a property of the node; and
creating an event processing query with respect to the property of the node.
14. The system of claim 11 , wherein:
the operator internally stores the state information for a plurality of different points in time; and
the graph query specifies a temporal constraint.
15. The system of claim 14 , wherein the state information comprises a temporal dimension.
16. The system of claim 11 , wherein the operations further comprise:
receiving user input that specifies a time period for storing the state information in the operator; and
modifying the operator to maintain the state information for a duration of the time period.
17. The system of claim 11 , wherein accessing the node comprises traversing a set of nodes in the object information data structure to identify the node of the set of nodes that corresponds to the operator, wherein the set of nodes in the object information data structure symbolically represents a corresponding set of operators in the data processing system.
18. The system of claim 11 , wherein the operations further comprise accessing the copy of the state information from the node by identifying a property of the node comprising the copy of the state information.
19. The system of claim 11 , wherein:
the data processing system comprises an event processing system;
the data source comprises an event source; and
the operator processes one or more events from the event source based on timestamps of the one or more events.
20. The system of claim 11 , wherein the object information data structure is an object graph that dynamically updates the copy of the state information in the node as changes to the state information in the operator are detected.