Systems and methods for highly parallel processing of parameterized simulations
Systems and associated methods for highly parallel processing of parameterized simulations are described. Embodiments permit processing of stochastic data-intensive simulations in a highly parallel fashion in order to distribute the intensive workload. Embodiments utilize methods of seeding records in a database with a source of pseudo-random numbers, such as a compressed seed for a pseudo-random number generator, such that seeded records may be processed independently in a highly parallel fashion. Thus, embodiments provide systems and associated methods facilitating quicker data-intensive simulation by enabling highly parallel asynchronous simulations.
1. A method comprising:
utilizing at least one processor to execute computer code configured to perform the steps of:
assigning records of a database for seeding to one of a plurality of parallel-processing nodes, wherein each of the records is assigned to a single of the plurality of parallel-processing nodes;
at each of and in parallel with the plurality of parallel-processing nodes, seeding each record using the node assigned to the record, said seeding further comprising adding a unique source of pseudo-random numbers to each record, wherein the unique source of pseudo-random numbers is stored within a corresponding record; and
providing, using a cycle of seeds, seeded records to a plurality of processing nodes for parallel asynchronous simulations, wherein the plurality of processing nodes is different from the plurality of parallel-processing nodes, wherein each of the seeded records is provided to a single of the plurality of processing nodes, wherein the providing comprises:
with respect to the cycle of seeds, generating skips of varied length based on the records of the database; and
modifying, based on the generated skips of varied length, a start point for a cycle of at least one simulation; and
each of the seeded records undergoing a single simulation, wherein the pseudo-random number of the corresponding record is utilized during the simulation.
2. The method according to claim 1 , wherein assigning records of a database for seeding further comprises assigning records to a plurality of processing nodes for seeding independently and asynchronously.
3. The method according to claim 1 , wherein the unique source of pseudo-random numbers further comprises a compressed seed for a pseudo-random number generator.
4. The method according to claim 1 , wherein the parallel asynchronous simulations further comprise, a plurality of simulations, each executed in accordance with a given seeded record by at least one of the plurality of processing nodes.
5. The method according to claim 1 , wherein adding a unique source of pseudo-random numbers to each record further comprises adding a source of pseudo-random numbers to each record enabling each of the plurality of processing nodes to generate pseudo-random numbers locally for a simulation.
6. The method according to claim 1 , wherein adding a unique source of pseudo-random numbers to each record further comprises providing a compressed seed to each record enabling each of the plurality of processing nodes to generate a disjoint sequence of pseudo-random numbers for the record using a skip-ahead technique for a single pseudo-random number generator locally.
7. The method according to claim 6 , wherein using a skip-ahead technique further comprises pre-computation of a set of skip parameters whose size is a logarithmic function of the cycle length of the pseudo-random number generator.
8. The method according to claim 1 , wherein providing a unique source of pseudo-random numbers to each record further comprises providing a compressed seed to each record enabling each of the plurality of processing nodes to locally generate a sequence of pseudo-random numbers for the record that, above a given probability, is disjoint from a sequence for all other records, using a technique based on multiple pseudo-random number generators.
9. The method according to claim 1 , further comprising receiving simulation results from parallel asynchronous simulations executed with the plurality processing nodes using the seeded records and recording the simulation results.
10. The method according to claim 1 , wherein said providing of seeded records comprises employing a single pseudo-random number generator to provide seeded records to a plurality of processing nodes for parallel asynchronous simulations and for performing at least one simulation.
11. The method according to claim 10 , wherein to provide seeded records to a plurality of processing nodes for parallel asynchronous simulations comprises seeding a record with a node number and intra-node record identifier.