System and method for management of processing task across plurality of processors
Disclosed is a system and method for management of a task across a plurality of processors, wherein the task processing of a plurality of records. The system includes a plurality of processors and an administrator module. Each of the plurality of processors processes a set of records added in a given time interval and the administrator module determines the time interval for a given processor. The administrator module is operable to receive a timestamp detailing a time when the given record was added to the database arrangement; determine a time period in which the plurality of records were added to the database arrangement; determine a time and record threshold for a given processor; and adjust the time interval allocated to each of the plurality of processors according to the time threshold and the record threshold, based on a time-based distribution of records in the database arrangement.
1. A system comprising:
a plurality of processors;
a database arrangement comprising a plurality of records; and
an administrator module configured to:
receive metadata associated with the plurality of records from the database arrangement, wherein corresponding metadata associated with each record comprises a timestamp detailing a time when each record was added to the database arrangement;
determine, from the corresponding metadata associated with each record of the plurality of records, a time period in which the plurality of records was added to the database arrangement;
determine, from the time period and a number of processors in the plurality of processors, a time threshold for each processor;
determine, from a number of records in the plurality of records and the number of processors in the plurality of processors, a record threshold for each processor;
divide the plurality of records into groups of records corresponding to each processor according to the record threshold for each processor; and
allocate the corresponding groups of records to each processor during the corresponding time threshold for each processor according to a time-based distribution of the plurality of records in the database arrangement, wherein the plurality of processors simultaneously modify the corresponding groups of records.
2. The system of claim 1 , wherein the administrator module extracts the corresponding metadata associated with each record of the plurality of records from each record.
3. The system of claim 1 , wherein the time period is within a tolerance range of the record threshold.
4. The system of claim 3 , wherein the tolerance range is in a range of 0.1 percent to 10 percent of the record threshold.
5. The system of claim 4 , wherein the tolerance range is equal to 5 percent.
6. The system of claim 1 , wherein the database arrangement and the administrator module are communicably coupled using a data communication network.
7. A method comprising:
using a database arrangement to communicate metadata associated with a plurality of records of the database arrangement to an administrator module; and
using the administrator module to:
receive the metadata associated with the plurality of records from the database arrangement, wherein corresponding metadata associated with a each record comprises a timestamp detailing a time when each record was added to the database arrangement;
determine, from the corresponding metadata associated with each record of the plurality of records, a time period in which the plurality of records was added to the database arrangement;
determine, from the time period and a number of processors in a plurality of processors, a time threshold for each processor;
determine, from the number of records in the plurality of records and the number of processors in the plurality of processors, a record threshold for each processor;
divide the plurality of records into groups of records corresponding to each processor according to the record threshold for each processor; and
allocate the corresponding groups of records to each processor during the corresponding time threshold for each processor according to a time-based distribution of the plurality of records in the database arrangement, wherein the plurality of processors simultaneously modify the corresponding groups of records.
8. The method of claim 7 , wherein the method comprises using the administrator module to extract the corresponding metadata associated with each record of the plurality of records from each record.
9. The method of claim 7 , wherein the time period is within a tolerance range of the record threshold.
10. The method of claim 9 , wherein the tolerance range is in a range of 0.1 percent to 10 percent of the record threshold.
11. The method of claim 10 , wherein the tolerance range is equal to 5 percent.