Coordinated processing of data by networked computing resources
Systems, methods, and computer-readable media for coordinating processing of data by multiple networked computing resources include monitoring data associated with a plurality of networked computing resources, and coordinating the routing of data processing segments to the networked computing resources.
1. A system for coordinating processing of data by multiple networked computing resources, the system comprising at least one processor configured to:
observe data associated with a plurality of networked computing resources, the observed data including parameters of a dynamically-varying delay on data processing segments for at least one of the plurality of networked computing resources;
receive from one or more data sources signals representing instructions for execution of at least one data process executable by the plurality of networked computing resources;
based on the observed data including the parameters of the dynamically-varying delay on data processing segments for the at least one of the plurality of networked computing resources:
divide the at least one data process into at least one data processing segment, each data processing segment to be routed to one of the plurality of networked computing resources, wherein dividing the at least one data process includes selecting to which of the plurality of networked computing resources at least one of the plurality of data processing segments is to be routed;
determine a plurality of timing parameters, each of the plurality of timing parameters to be associated with a corresponding one of the plurality of data processing segments, the plurality of timing parameters determined to cause coordinated execution of the at least one data processing segment by the plurality of networked computing resources based at least in part on the parameters of the dynamically-varying delay; and
route the at least one data processing segment to the plurality of corresponding networked computing resources in a timing sequence based on the timing parameters.
2. The system of claim 1 , wherein dividing the at least one data process into at least one data processing segment includes: for each of the selected networked computing resources, determining a size of the corresponding at least one data processing segments.
3. The system of claim 1 , dividing the at least one data process into at least one data processing segment includes or determining the plurality of timing parameters is based on a type of the at least one data process.
4. The system of claim 1 , wherein the at least one processor is configured to: determine whether liquidity in addition to posted liquidity is historically available at a particular networked computing resource,
and wherein dividing the at least one data process includes dividing the at least one data process into at least two data processing segments to be routed to the particular networked computing resource, the at least two data processing segments having timing parameters to target the posted available trade liquidity and the additional liquidity.
5. The system of claim 1 , wherein observing the data associated with the plurality of networked computing resources includes:
determining components of a latency associated with routing a data processing segment to one of the plurality of networked computing resources,
wherein determining the components of the latency is based on differences in latencies associated with routing different types of data processing segments to the one of the plurality of networked computing resources.
6. The system of claim 5 , wherein the components of the latency include at least one range of delay lengths introduced by at least one of the plurality of networked computing resources.
7. The system of claim 1 , wherein dividing the at least one data process into at least one data processing segment includes or determining the plurality of timing parameters is based on historical execution latencies for the plurality of networked computing resources.
8. The system of claim 7 , wherein dividing the at least one data process into at least one data processing segment includes or determining the plurality of timing parameters is based on a volatility level and the historical execution latencies.
9. A method for coordinating processing of data by multiple networked computing resources, the method comprising:
observing data associated with a plurality of networked computing resources, the observed data including parameters of a dynamically-varying delay on data processing segments for at least one of the plurality of networked computing resources;
receiving from one or more data sources signals representing instructions for execution of at least one data process executable by the plurality of networked computing resources;
based on the observed data including the parameters of the dynamically-varying delay on data processing segments for the at least one of the plurality of networked computing resources:
dividing the at least one data process into at least one data processing segment, each data processing segment to be routed to one of the plurality of networked computing resources, wherein dividing the at least one data process includes selecting to which of the plurality of networked computing resources at least one of the plurality of data processing segments is to be routed;
determining a plurality of timing parameters, each of the plurality of timing parameters to be associated with a corresponding one of the plurality of data processing segments, the plurality of timing parameters determined to cause coordinated execution of the at least one data processing segment by the plurality of networked computing resources based at least in part on the parameters of the dynamically-varying delay; and
routing the at least one data processing segment to the plurality of corresponding networked computing resources in a timing sequence based on the timing parameters.
10. The method of claim 9 , wherein dividing the at least one data process into at least one data processing segment includes: for each of the selected networked computing resources, determining a size of the corresponding at least one data processing segments.
11. The method of claim 9 , wherein dividing the at least one data process into at least one data processing segment includes or determining the plurality of timing parameters is based on a type of the at least one data process.
12. The method of claim 9 , wherein the method includes:
determining whether liquidity in addition to posted liquidity is historically available at a particular networked computing resource,
and wherein dividing the at least one data process includes dividing the at least one data process into at least two data processing segments to be routed to the particular networked computing resource, the at least two data processing segments having timing parameters to target the posted available trade liquidity and the additional liquidity.
13. The method of claim 9 , wherein observing the data associated with the plurality of networked computing resources includes:
determining components of a latency associated with routing a data processing segment to one of the plurality of networked computing resources,
wherein determining the components of the latency is based on differences in latencies associated with routing different types of data processing segments to the one of the plurality of networked computing resources.
14. The method of claim 13 , wherein the components of the latency include at least one range of delay lengths introduced by at least one of the plurality of networked computing resources.
15. The method of claim 9 , wherein dividing the at least one data process into at least one data processing segment includes or determining the plurality of timing parameters is based on historical execution latencies for the plurality of networked computing resources.
16. The method of claim 15 , wherein dividing the at least one data process into at least one data processing segment includes or determining the plurality of timing parameters is based on a volatility level and the historical execution latencies.
17. A non-transitory computer-readable medium having stored thereon instructions which, when executed by at least one processor, configure the at least one processor to:
observe data associated with a plurality of networked computing resources, the observed data including parameters of a dynamically-varying delay on data processing segments for at least one of the plurality of networked computing resources;
receive from one or more data sources signals representing instructions for execution of at least one data process executable by the plurality of networked computing resources;
based on the observed data including the parameters of the dynamically-varying delay on data processing segments for the at least one of the plurality of networked computing resources:
divide the at least one data process into at least one data processing segment, each data processing segment to be routed to one of the plurality of networked computing resources, wherein dividing the at least one data process includes selecting to which of the plurality of networked computing resources at least one of the plurality of data processing segments is to be routed;
determine a plurality of timing parameters, each of the plurality of timing parameters to be associated with a corresponding one of the plurality of data processing segments, the plurality of timing parameters determined to cause coordinated execution of the at least one data processing segment by the plurality of networked computing resources based at least in part on the parameters of the dynamically-varying delay; and
route the at least one data processing segment to the plurality of corresponding networked computing resources in a timing sequence based on the timing parameters.
18. The non-transitory computer-readable medium of claim 17 , wherein dividing the at least one data process into at least one data processing segment includes or determining the plurality of timing parameters is based on a volatility level and on historical execution latencies for the plurality of networked computing resources.
19. The non-transitory computer-readable medium of claim 17 , wherein observing the data associated with the plurality of networked computing resources includes:
determining components of a latency associated with routing a data processing segment to one of the plurality of networked computing resources,
wherein determining the components of the latency is based on differences in latencies associated with routing different types of data processing segments to the one of the plurality of networked computing resources.
20. The non-transitory computer-readable medium of claim 19 , wherein the components of the latency include at least one range of delay lengths introduced by at least one of the plurality of networked computing resources.