Massively distributed processing system with modular client agent and associated method
View Patent ↗A massively distributed processing system and associated methods are disclosed that provide a processing architecture for utilizing a multitude of widely distributed devices to process distributed project workloads. To provide the processing infrastructure, a modular client agent program, which may include a system component and a separate project component, operates on the distributed devices to process workloads. For different projects, different project components may be provided to run on the base system component. In addition, a device capabilities database and an incentive database can be used by the server system to facilitate operations and encourage participation by client systems. Other databases may also be utilized to enhance system operations and functionality. And a wide variety of applications are possible, including network site testing, network site indexing, distributed data back-up, file sharing, data caching, data conversion, and scientific research, as well as many other distributed projects.
1. A massively distributed processing system comprising:
a server coupled by a network to a multitude of distributed devices capable of processing workloads for the massively distributed processing system;
a client agent program distributed to selected distributed devices within the multitude of distributed devices in response to an agreement by the selected distributed devices to process workloads for the massively distributed processing system;
a processing system component within the client agent program having a user interface for communicating with a user of each of the selected distributed devices, a communications interface for communicating with the server, and a core agent module, the core agent module configured to manage operations of the client agent program; and
a plurality of project components managed by the core agent prepam module and each having at least one task module with one or more workload units for processing workloads for the massively distributed processing system.
2. The system of claim 1 , wherein the server comprises:
a server communication interface for communicating with the multitude of distributed devices;
a database system coupled to the server with a database interface; and
a task module and work unit manager for distributing the task modules and the workload units to the selected distributed devices in response to workload processing capabilities of the selected distributed devices.
3. The system of claim 2 , wherein the database system comprises a workload capability database and a device capability database.
4. The system of claim 3 , wherein the workload capability database comprises workload factors describing capabilities necessary for the selected distributed devices to process woridoads for the massively distributed processing system.
5. The system of claim 3 , wherein the device capabilities database comprises device capability factors defining an ability of the selected distributed devices to process workloads for the massively distributed processing system.
6. The system of claim 3 , wherein the server distributes a particular workload to a particular one of the selected distributed devices in response, in at least part, to comparing workload capability factors for the particular workload to device capability factors for the particular one of the selected distributed devices.
7. The system of claim 2 , wherein the server further comprises a version and phase control module.
8. The system of claim 7 , wherein the version and phase control module ensures that a workload for a particular task module sent to a client agent program in one of the selected distributed device includes a most recent version of the particular task module.
9. The system of claim 7 , wherein the version and phase control module ensures that a test version of a task module is sent only to ones of the selected distributed devices configured to execute the test version of the task module.
10. The system of claim 1 further comprising a security subsystem and security interface.
11. The system of claim 10 , wherein the server is coupled to the security subsystem with an internet interface.
12. The system of claim 11 , wherein the processing system component is coupled to the server via the security subsystem and the internet interface.
13. The system of claim 1 , wherein the agreement by the selected distributed devices to process workloads for the massively distributed processing system is generated in response to an incentive offered to the selected distributed devices by the server.
14. The system of claim 13 , wherein the incentive is generated in response to one of the selected distributing devices completing a workload.
15. The system of claim 13 , wherein the incentive is generated in response to device capability factors of one of the selected distributing devices.
16. The system of claim 1 , wherein multiple task modules in one of the selected distributed devices process different workloads concurrently.
17. A method of operating a massively distributed processing system comprising the steps of:
coupling a server coupled by a network to a multitude of distributed devices capable of processing workloads for the massively distributed processing system;
loading a client agent program distributed to selected distributed devices within the multitude of distributed devices in response to an agreement by the selected distributed devices to process workloads for the massively distributed processing system;
operating a processing system component within the client agent program having a user interface for communicating with a user of each of the selected distributed devices, a communications interface for communicating with the server, and a core agent module, the core agent module configured to manage operations of the client agent program; and
operating a plurality of project components managed by the core agent program and each having at least one task module with one or more workload units for processing workloads for the massively distributed processing system.
18. The method of claim 17 , wherein the server comprises:
a server communication interface for communicating with the multitude of distributed devices;
a database system coupled to the server with a database interface; and
a task module and work unit manager for distributing the task modules and the workload units to the selected distributed devices in response to workload processing capabilities of the selected distributed devices.
19. The method of claim 18 , wherein the database system comprises a workload capability database and a device capability database.
20. The method of claim 19 , wherein the workload capability database comprises workload factors describing capabilities necessary for the selected distributed devices to process workloads for the massively distributed processing system.
21 .The method of claim 19 , wherein the device capabilities database comprises device capability factors defining an ability of the selected distributed devices to process workloads for the massively distributed processing system.
22. The method of claim 19 , wherein the server distributes a particular workload to a particular one of the selected distributed devices in response, in at least part, to comparing workload capability factors for the particular workload to device capability factors for the particular one of the selected distributed devices.
23. The method of claim 18 , wherein the server further comprises a version and phase control module.
24. The method of claim 23 , wherein the version and phase control module ensures that a workload for a particular task module sent to a client agent program in one of the selected distributed devices includes a most recent version of the particular task module.
25. The method of claim 23 , wherein the version and phase control module ensures that a test version of a task module is sent only to ones of the selected distributed devices configured to execute the test version of the task module.
26. The method of claim 17 , further comprising a security subsystem and security interface.
27. The method of claim 26 , wherein the server is coupled to the security subsystem with an internet interface.
28. The method of claim 26 , wherein the processing system component is coupled to the server via the security subsystem and the internet interface.
29. The method of claim 17 , wherein the agreement by the selected distributed devices to process workloads for the massively distributed processing system is generated in response to an incentive offered to the selected distributed devices by the server.
30. The method of claim 29 , wherein the incentive is generated in response to one of the selected distributing devices completing a workload.
31. The method of claim 29 , wherein the incentive is generated in response to device capability factors of one of the selected distributing devices.
32. The method of claim 17 , wherein multiple task modules in one of the selected distributed devices process different workloads concurrently.