System for allocating computing resources of distributed computer system with transaction manager
In one aspect, the present invention features a system including a local computing device in communication with a distributed computing system, the local computing device configured to perform computations for a first portion of a computer software application and to send a second portion of the application for computation on the distributed computing system. According to such an aspect, the distributed computing system may include a node computing device configured to perform computations for the second portion of the application, a persistent data storage queue in communication with the node computing device, the persistent data storage configured to store the second portion of the application, wherein a minimum availability of the distributed computing system is defined by an availability of the persistent data storage, and a compute function deployed on the distributed computing system, the second portion of the application including a job with an input, wherein the input has a task to be performed by the compute function, and wherein the input need not be supplied to the job at a time of job creation.
1. A system, comprising:
a processor;
a transaction manager in communication with a plurality of local computers, wherein said transaction manager supports a plurality of message protocols to enable communication between said transaction manager and said plurality of local computers, wherein each said local computer runs a calling application;
a queue in communication with said transaction manager, wherein said queue receives and stores a plurality of jobs and task inputs from said transaction manager, sends a plurality of computation requests to one or more of a plurality of node computers, and provides computation results when polled by said transaction manager;
a scheduler in communication with said queue, wherein said scheduler routes a plurality of incoming tasks to a plurality of compute functions on the plurality of node computers and allocates computing resources of said plurality of node computers,
a service manager in communication with said transaction manager, said scheduler, and said plurality of node computers, wherein said service manager controls allocation of computing resources among a plurality of users of said plurality of local computers such that a failure of one of said calling applications of one of said users will not adversely affect another of said calling applications of another of said users, even if both applications are running simultaneously; and
a cache in communication with said plurality of node computers, wherein a minimum availability of said system is defined by an availability of said queue such that addition of one or more devices to said system downstream of said queue does not increase a failure probability of said system.
2. The system of claim 1 , wherein said transaction manager facilitates delivery of each of said plurality of computation requests from said calling applications to said system.
3. The system of claim 2 , wherein said transaction manager guarantees delivery of each of said plurality of computation requests from said calling applications to said system.
4. The system of claim 1 , wherein each of said calling applications is created in a different programming language.
5. The system of claim 1 , wherein communications between said transaction manager and each of said plurality of local computers are secure and involve an authentication process before access to said system is granted.
6. The system of claim 1 , wherein all information pertinent for a particular one of said plurality of jobs is stored persistently in said queue at least until said job has been completed or has expired.