Deterministic message processing in a distributed network

1 . A system for message processing in a distributed network, the system comprising:

first and second computer systems coupled to one another in the distributed network, each of the first and second computer systems comprising at least one processor and at least one memory in which computer-executable instructions are stored that, when executed by the at least one processor, implement a message operator configured to process messages arriving at the message operator to manage a data structure having a state deterministically established via processing of the messages;

wherein the message operator of the first computer system is configured to, prior to attempting to process unprocessed messages, reroute the unprocessed messages received by the message operator of the first computer system to the message operator of the second computer system;

wherein the message operator of the second computer system is configured to establish, based on the rerouted unprocessed messages, an order of an input message stream and process the input message stream in the order to establish a current state of the data structure arising from ordered processing of the input message stream,

wherein the message operator of the second computer system is configured to, once the current state is established, send the input message stream to the message operator of the first computer system in accordance with the established order;

wherein the message operator of the first computer system is configured to recreate, via reception and processing of the input message stream in the order, the current state of the data structure,

wherein the message operators of the first and second computer systems are deterministic such that the processing of the input message stream by the message operators is deterministic, and

wherein the deterministic processing of the message operators of the first and second computer systems causes the data structures for the first and second computer systems to reach a same state given that both of the message operators process the input message stream in the established order.

2 . The system of claim 1 , wherein each message is representative of an event such that the input message stream is representative of an input message event stream.

3 . The system of claim 1 , wherein each message operator comprises a message gateway configured to receive and transmit the messages, and further comprises an event processing engine configured to manage the data structure in accordance with the messages received by the message gateway.

4 . The system of claim 3 , wherein:

each of the first and second computer systems comprises a plurality of server computers;

the message gateway is implemented on a first server computer of the plurality of server computers; and

the event processing engine is implemented on a second server computer of the plurality of server computers.

5 . The system of claim 3 , wherein the message gateway of the message operator of the second computer system is configured to send the messages of the input message stream to the message operator of the first computer system once the event processing engine confirms processing of the messages of the input message stream to establish the current state of the data structure.

6 . The system of claim 1 , wherein:

the message operator of each of the first and second computer systems is a first message operator of a plurality of message operators implemented on each of the first and second computer systems;

the first message operators are configured to process messages regarding events in a first field; and

second message operators of the plurality of message operators implemented on each of the first and second computer systems are configured to process messages regarding events in a second field.

7 . The system of claim 1 , wherein:

the message operator of the second computer system implements a leader role for the message processing, and the message operator of the first computer system implements a follower role for the message processing; and

the message operator of the second computer system switches assignment of the leader role and the follower role upon notification of the message operator of the first computer system.

8 . The system of claim 7 , wherein the assignment is switched via an in-memory designation associating the message operator as implementing either the leader role or the follower role.

9 . The system of claim 7 , wherein the assignment of the leader role is made in accordance with which of the first and second computer systems is associated with a lead venue for the input message stream.

10 . The system of claim 9 , wherein each message is an order for a financial product such that the active venue corresponds with a venue at which the financial product is actively trading.

11 . The system of claim 1 , wherein:

execution of the computer-executable instructions further implements a publication generator of each of the first and second computer systems, the publication generator being configured to generate a publication indicative of the current state of the data structure;

the message operator of the second computer system incorporates a time stamp into each one of the messages of the input message stream when sending the messages to the message operator of the first computer system;

the publication generator of the first computer system is configured to use the time stamp to determine whether to include the message in the publication.

12 . The system of claim 11 , wherein:

the message operator of the second computer system is configured to send a time beacon indicative of a time at which the publication is generated;

the publication generator of the first computer system is configured to determine whether to include the message in the publication based on whether the time stamp of the message precedes the time beacon.

13 . The system of claim 11 , wherein the publication generator is implemented on a further server computer of each of the first and second computer systems.

14 . A computer implemented method of message processing in a distributed network, the method comprising:

receiving, by a first message operator comprising at least one processor, an incoming message;

processing, by the first message operator, the received incoming message to establish an order of an input message stream and manage a first data structure having a state deterministically established via processing of the input message stream; and

forwarding, by the first message operator via the distributed network, regardless of whether the first message is matched by the processing and once the state is established, the input message stream to a second message operator in accordance with the established order for processing of the input message stream;

recreating, via reception and processing of the input message stream, the state in a second data structure managed by the second message operator;

wherein the first and second message operators are deterministic, the processing of the input message stream by the first and second message operators being deterministic based on the first and second message operators being deterministic, and

wherein, after the deterministic processing of the first and second message operators, the first and second data structures having a same state given that both of the first and second message operators process the input message stream in the established order.

15 . The method of claim 14 , further comprising receiving, by a message gateway of the first message operator, confirmation of the processing of the message by an event processing engine of the first message operator, wherein forwarding the message is implemented after the confirmation is received.

16 . The method of claim 14 , wherein forwarding the message comprises forwarding the message with a time stamp, the second message operator being configured to use the time stamp for generation of a publication indicative of the state of the data structure.

17 . The method of claim 14 , wherein the message regards events in a first field, and wherein the method includes reversing roles of the first and second message operators for messages regarding events in a second field relative to roles of first and second message operators for messages regarding events in the first field.

18 . A computer implemented method of message processing in a distributed network in which a lead message operator and a shadow message operator are disposed, each of the lead message operator and the shadow message operator comprising at least one processor, the method comprising:

receiving, by the shadow message operator, an incoming message;

at a time that the incoming message is not from the lead message operator and prior to attempting to process the incoming message, rerouting the incoming message to the lead message operator for processing to establish an order of an input message stream and to manage a lead data structure having a state deterministically established via processing of the input message stream;

at a time that the incoming message is from the lead message operator, processing, by the shadow message operator, the incoming message to manage a shadow data structure in accordance with the established order to recreate the state of the lead data structure established as a result of processing of the input message stream by the lead message operator;

wherein the lead and shadow message operators are deterministic, the processing of the input message stream by the lead and shadow message operators being deterministic based on the lead and shadow message operators being deterministic, and

wherein, after the deterministic processing of the lead and shadow message operators, the lead and shadow data structures having a same state given that both of the lead and shadow message operators process the input message stream in the established order.

19 . The method of claim 18 , further comprising, at a time that the incoming message is from the lead message operator, analyzing a timestamp in the message to determine when to generate a publication indicative of the state of the shadow data structure.

20 . The method of claim 18 , wherein the incoming message regards events in a first field, and wherein roles of the lead and shadow message operators are reversed for messages regarding events in a second field.