System and method for processing information via networked computers including request handlers, process handlers, and task handlers
Systems and methods for processing information via networked computers leverage request handlers, process handlers, and task handlers to provide efficient distributed processing of processing jobs. A request handler can receive service requests for processing jobs, process handlers can identify tasks to be performed in connection with the processing jobs, and task handlers can perform the identified tasks, where the request handler, the process handlers, and the task handlers can be distributed across a plurality of networked computers.
1. A system for applying a distributed computing architecture to enhance task processing with high availability computing power, the system comprising:
a decentralized distributed computing network that executes a processing job in a distributed manner, wherein the decentralized distributed computing network comprises a plurality of networked computers, the plurality of networked computers comprising a request handler, a plurality of process handlers, and a plurality of task handlers, the process handlers being resident on a plurality of different networked computers, the task handlers being resident on a plurality of different networked computers, the processing job having an associated process flow, the process flow including (1) a plurality of tasks without a specification as to which of the networked computers are to perform the tasks and (2) logic that defines a relationship between the tasks;
wherein the decentralized computing network is configured to distributively execute the processing job via a plurality of operations, wherein the operations comprise:
the request handler receiving a service request for the processing job;
the decentralized distributed computing network storing state information for the processing job;
at least one process handler among the plurality of process handlers volunteering for servicing the processing job corresponding to the received service request based on its availability for servicing the processing job, the at least one process handler thus being a volunteer process handler;
the volunteer process handler (1) analyzing the state information for the processing job to determine whether any tasks in the process flow remain to be performed based on the logic for the process flow, (2) in response to the state information analysis indicating that a task remains for the process flow, identifying a task from the process flow to be performed, and (3) in response to the state information analysis indicating that no tasks remain for the process flow, determining that the processing job has been completed;
for each of a plurality of the identified tasks, at least one task handler among the plurality of task handlers volunteering for performing that identified task based on it being able to perform that identified task, the at least one task handler thus being a volunteer task handler;
each volunteer task handler performing an identified task to generate a task result; and
the decentralized distributed computing network storing updated state information for the processing job, the updated stored state information being based on the task results.
2. The system of claim 1 wherein the decentralized computing network is further configured to repeat the operations with respect to a plurality of service requests for a plurality of processing jobs to provide distributed processing by the decentralized distributed computing network of the processing jobs such that a plurality of the process handlers serve as volunteer process handlers for the processing jobs and a plurality of the task handlers serve as volunteer task handlers for the processing jobs.
3. The system of claim 1 wherein the operations further comprise:
the request handler and the volunteer process handler redundantly storing portions of the state information for the processing job.
4. The system of claim 1 wherein the operations further comprise:
the request handler communicating a volunteer request for a processing job to a plurality of the process handlers; and
a plurality of the process handlers deciding whether to volunteer for servicing the processing job corresponding to the communicated volunteer request based on the communicated volunteer request.
5. The system of claim 1 wherein the operations further comprise:
the volunteer process handler communicating a volunteer request for an identified task to a plurality of the task handlers; and
a plurality of the task handlers deciding whether to volunteer for performing the identified task corresponding to the communicated volunteer request based on the communicated volunteer request.
6. The system of claim 1 wherein a plurality of the networked computers on which a plurality of the task handlers reside have different resources, and wherein the operations further comprise:
a plurality of the task handlers deciding whether to volunteer for performing the identified tasks as a function of the resources of the networked computers on which the task handers are resident.
7. The system of claim 1 wherein each of a plurality of the networked computers comprises a process handler and a task handler.
8. The system of claim 1 wherein the operations further comprise:
the request handler assigning a global unique identifier (GUID) to the processing job;
the request handler determining whether a recovery procedure is to be initiated for the processing job; and
in response to a determination that a recovery procedure is to be initiated for the processing job, the request handler generating a recovery request message that includes the GUID.
9. The system of claim 1 wherein a plurality of the networked computers are arranged as a plurality of processing regions, each of a plurality of the processing regions comprising a plurality of the process handlers and a plurality of the task handlers, and wherein the operations further comprise:
the volunteer process handler limiting solicitation of volunteer task handlers to only task handlers within its own processing region.
10. A system for applying a distributed computing architecture to enhance task processing with high availability computing power, the system comprising:
a plurality of networked computers that are arranged as a decentralized distributed computing network to distributively process a plurality of processing jobs in accordance with a plurality of process flows that are associated with the processing jobs, wherein each of a plurality of the process flows comprises (1) a plurality of tasks for that process flow without a specification as to which of the networked computers are to perform the tasks, and (2) logic that defines a relationship between that process flow's tasks;
wherein (1) each of a first plurality of the networked computers comprises a request handler deployed thereon, (2) each of a second plurality of the networked computers comprises a process handler deployed thereon, and (3) each of a third plurality of the networked computers comprises a task handler deployed thereon such that a plurality of the request handlers, process handlers, and task handlers are distributed among the networked computers;
wherein the decentralized distributed computing network is further configured to allocate work for the processing jobs among a plurality of volunteers in a decentralized manner, wherein the volunteers comprise a plurality of volunteers among the process handlers and a plurality of volunteers among the task handlers:
wherein a plurality of the request handlers are configured to receive a plurality of service requests for a plurality of the processing jobs;
wherein the decentralized distributed computing network is further configured to store state information for the processing jobs corresponding to the received service requests;
wherein each of a plurality of the volunteer process handlers is configured to (1) analyze (i) the state information for a processing job and (ii) the logic for the process flow associated with that processing job to determine whether any tasks remain to be performed for that processing job, (2) in response to the analysis indicating that a task remains for that processing job, identify a task to be performed, and (3) in response to the analysis indicating that no tasks remain for that processing job, determine that that processing job has been completed;
wherein each of a plurality of the volunteer task handlers is configured to perform an identified task such that a plurality of the volunteer task handlers generate a plurality of task results; and
wherein the decentralized distributed computing network is further configured to store updated state information for the processing jobs, the updated stored state information being based on the task results.
11. The system of claim 10 wherein each of a plurality of the process handlers is configured to volunteer for servicing the processing jobs based on their availabilities for servicing the processing jobs.
12. The system of claim 11 wherein each of a plurality of the task handlers is configured to volunteer for tasks of the processing jobs based on their being able to perform the tasks.
13. The system of claim 10 wherein each of a plurality of the task handlers is configured to volunteer for tasks of the processing jobs based on their being able to perform the tasks.
14. The system of claim 10 wherein each of a plurality of the request handlers is configured to generate a plurality of volunteer requests for servicing the processing jobs; and
wherein each of a plurality of the process handlers is further configured to (1) read the generated volunteer requests and (2) decide whether to volunteer for servicing process jobs based on the read volunteer requests.
15. The system of claim 14 wherein each of a plurality of the request handers is further configured to select a volunteer process handler for servicing a processing job from among a plurality of volunteer process handlers for that processing job.
16. The system of claim 15 wherein each of a plurality of the request handlers is further configured to send the generated volunteer requests to a plurality of the process handlers for reading thereby.
17. The system of claim 10 wherein each of a plurality of the volunteer process handlers is further configured to generate a plurality of volunteer requests for performing identified tasks; and
wherein each of a plurality of the task handlers is further configured to (1) read the generated volunteer requests and (2) decide whether to volunteer for tasks of the processing jobs based on the read volunteer requests.
18. The system of claim 10 wherein a plurality of members of the second plurality of the networked computers are also members of the third plurality of the networked computers.
19. The system of claim 10 wherein the decentralized distributed computing network is further configured to organize a plurality of the networked computers into a plurality of processing regions, each processing region comprising a plurality of the first and second plurality of the networked computers.
20. The system of claim 19 wherein the decentralized distributed computing network is further configured to limit the volunteer process handlers and the volunteer task handlers for a processing job to only those process handlers and task handlers that reside in the same processing region.