Systems and methods to process a request received at an application program interface

1. A system to process a request, the system comprising:

at least one processor in communication with a non-transitory machine-readable medium storing instructions that, when executed by the at least one processor, cause the at least one processor to receive, over a network, the request from a client machine, the request comprising a job that is associated with data, the request received at an application program interface, the job indicating a set of tasks to be performed;

a peer-to-peer network of processing modules configured to generate a plurality of sub-jobs by dividing the job into equal quantities of work, the job being utilized to retrieve transaction data from a network-based marketplace, the plurality of sub-jobs respectively comprising at least one task of the set of tasks to be performed that corresponds to a portion of the data, the peer-to-peer network of processing modules configured to schedule the plurality of sub-jobs for parallel processing based on an availability of resources that are respectively utilized by the sub-jobs, the peer-to-peer network of processing modules configured to parallel process the plurality of sub-jobs and to generate task results that are respectively associated with the plurality of sub-jobs and to move one or more sub-jobs to a sub-job retry queue based on an availability of resources, moving the one or more sub-jobs to the sub-job retry queue making the one or more sub-jobs visible to one or more processing modules of the peer-to-peer network to which the one or more sub-jobs were not previously visible; and

a pool of database servers, the peer-to-peer network of processing modules limits a number of sub-jobs that are scheduled based on a peak load associated with the pool of database servers.

2. The system of claim 1 , wherein the job is utilized to add listings on a network-based marketplace, and wherein the peer-to-peer network of processing modules is to communicate a first request for listing data to the client machine based on a first sub-job that is associated with listing data for a first plurality of listings, and wherein the peer-to-peer network of processing modules is to communicate a second request for listing data to the client machine based on a second sub-job that is associated with listing data for a second plurality of listings and wherein the peer-to-peer network of processing modules is further to automatically add the first plurality of listings and the second plurality of listings to the network-based marketplace.

3. The system of claim 1 , wherein the system includes a network-based marketplace and wherein the job is utilized to communicate transaction data from the network-based marketplace to the client machine and wherein the peer-to-peer network of processing modules is to communicate the transaction data for a first plurality of transactions to the client machine and to communicate the transaction data for a second plurality of transactions to the client machine.

4. The system of claim 1 , wherein the resources include local resources and the peer-to-peer network of processing modules is configured to schedule the plurality of sub-jobs based on an availability of local resources that are utilized by the sub-jobs.

5. The system of claim 1 , wherein the resources include remote resources and the peer-to-peer network of processing modules is configured to schedule the plurality of sub-jobs based on an availability of remote resources that are utilized by the sub-jobs.

6. The system of claim 1 , wherein the peer-to-peer network of processing modules is configured to compute a priority of the job.

7. The system of claim 1 , wherein the peak load associated with the pool of database servers is caused by a utilization of the data base servers by an entity other than the peer-to-peer network.

8. The system of claim 1 , wherein the peer-to-peer network of processing modules identifies a database server that is not accessible and retries to access the database server.

9. The system of claim 1 , wherein the peer-to-peer network of processing modules gathers task results into a job result.

10. A method to process a request, the method comprising:

receiving, over a network, a request from a client machine, the request comprising a job that is associated with data, the request received at an application program interface, the job indicating a set of tasks to be performed;

generating a plurality of sub-jobs by dividing the job into equal quantities of work, the job being utilized for retrieving transaction data from a network-based marketplace, the plurality of sub-jobs respectively comprising at least one task of the set of tasks to be performed that corresponds to a portion of the data, the generating performed by a processing node in a peer-to-peer network of processing nodes;

scheduling the plurality of sub-jobs for parallel processing based on an availability of resources that are respectively utilized by the sub-jobs, the resources including a pool of database servers, the scheduling including limiting a number of sub-jobs that are scheduled responsive to an identifying of a peak load associated with the pool of database servers, the scheduling being performed by a processing node in a peer-to-peer network of processing nodes

moving one or more sub-jobs of the plurality of sub-jobs to a sub-job retry queue based on an availability of resources, moving the one or more sub-jobs to the sub-job retry queue making the one or more sub-jobs visible to one or more processing nodes of the peer-to-peer network to which the one or more sub-jobs were not previously visible; and

parallel processing the sub-jobs with a plurality of processors to generate task results that are respectively associated with the plurality of sub-jobs, the parallel processing performed by at least one processing node in the peer-to-peer network of processing nodes.

11. The method of claim 10 , wherein the job is utilized to add listings on a network-based marketplace, the method of claim 10 further comprising:

communicating a first request for listing data from the network-based marketplace to the client machine based on a first sub-job that is associated with listing data for a first plurality of listings;

communicating a second request for listing data from the network-based marketplace to the client machine based on a second sub-job that is associated with listing data for a second plurality of listings; and

automatically adding the first plurality of listings and the second plurality of listings to the network-based marketplace.

12. The method of claim 10 , wherein the receiving of the request includes receiving a request to retrieve transaction data, and wherein the sub-jobs associated with the job include a first sub-job to retrieve transaction data for a first plurality of transactions, and a second sub-job to retrieve transaction data for a second plurality of transactions, and wherein the parallel processing of the sub-jobs includes:

communicating the transaction data for the first plurality of transactions to the client machine; and

communicating the transaction data for the second plurality of transactions to the client machine.

13. The method of claim 10 , further comprising adding a node to the peer-to-peer network of processing nodes to increase processing and storage capacity of the peer-to-peer network.

14. The method of claim 10 , further comprising removing a node from the peer-to-peer network of processing nodes to decrease processing and storage capacity of the peer-to-peer network.

15. The method of claim 10 , wherein the resources include local resources and wherein the scheduling of the plurality of sub-jobs is based on an availability of local resources that are utilized by the sub-jobs.

16. The method of claim 10 , wherein the resources include remote resources wherein the scheduling of the plurality of sub-jobs is based on an availability of remote resources that are utilized by the sub-jobs.

17. The method of claim 10 , wherein the peak load associated with the pool of database servers is caused by a utilization of the database servers by an entity that utilizes the pool of database servers other than the peer-to-peer network.

18. The method of claim 10 , further comprising identifying a pool of database servers that is not accessible and retrying to access the pool of database servers responsive to the identifying.

19. The method of claim 10 , further including gathering task results into a job result, the gathering performed by a processing node in the peer-to-peer network of processing nodes.

20. A non-transitory machine-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform the following actions:

receive, over a network, a request from a client machine, the request comprising a job that is associated with data, the request received at an application program interface, the job indicating a set of tasks to be performed;

generate a plurality of sub-jobs by dividing the job into equal quantities of work, the job is utilized to retrieve transaction data from a network-based marketplace, the plurality of sub-jobs respectively comprising at least one task of the set of tasks to be performed that corresponds to a portion of the data;

schedule the plurality of sub-jobs for parallel processing based on an availability of resources that are respectively utilized by the sub-job, the resources include a pool of database servers, the schedule of the plurality of jobs includes a limitation of a number of sub-jobs that are scheduled responsive to an identification of a peak load associated with the pool of database servers

move one or more sub-jobs of the plurality of sub-jobs to a sub-job retry queue based on an availability of resources, moving the one or more sub-jobs to the sub-job retry queue making the one or more sub-jobs visible to one or more processing nodes of the peer-to-peer network to which the one or more sub-jobs were not previously visible; and

parallel process the sub-jobs with a plurality of processors to generate task results that are respectively associated with the sub-jobs.

21. The method of claim 10 further comprising computing a priority of the job prior to the scheduling the priority of sub-jobs and wherein the scheduling of the plurality of sub-jobs is based on the priority of the job.