Intelligent service distributor and controller with real time service calibration
View Patent ↗Described embodiments provide real time load balancing and calibration processes. An aspect includes receiving a request to execute a service, assessing capacity of a target server to execute the service, and setting a threshold for a number of services to be executed on the target server. A further aspect includes assigning a unique identifier of the service to the target server, and based on the threshold and an assessment of a capability of other servers, assigning the service to the server.
1. A method for providing real time load balancing and calibration processes comprising:
receiving a request to execute a service;
assessing capacity of a target server to execute the service by at least calibrating the target server with respect to the service with parallel threads as a function of CPU consumption and RAM consumption;
setting a threshold for a number of services to be executed on the target server;
assigning a unique identifier of the service to the target server;
based on the threshold and an assessment of a capability of other servers, assigning at least a portion of the service to the target server;
receiving a response time, percentage of CPU usage, and free RAM of the target server;
increasing the number of jobs assigned to the service on the target server in specified intervals to reach an amount of work expected from the target server;
calculating expected response time for the target server based on a single thread single job process for the number of jobs increased;
increasing parallel processes for the target server;
calculating expected response time from the increase in parallel processes; and
discontinuing the increasing when the response time varies more than a specified percentage from the expected response time.
2. The method of claim 1 , wherein the threshold includes a maximum number of services and threads to be assigned.
3. The method of claim 1 , further comprising registering servers having the same services to be run in parallel.
4. The method of claim 1 , wherein responsive to assessment of the capability of other servers, the method includes redistributing work among the other servers as a function of a target time of completion for work running on the servers.
5. A system comprising:
a memory comprising computer-executable instructions; and
a processor executing the computer-executable instructions, the computer-executable instructions when executed by the processor cause the processor to perform operations comprising:
receiving a request to execute a service;
assessing capacity of a target server to execute the service by at least calibrating the target server with respect to the service with parallel threads as a function of CPU consumption and RAM consumption;
setting a threshold for a number of services to be executed on the target server;
assigning a unique identifier of the service to the target server;
based on the threshold and an assessment of a capability of other servers, assigning at least a portion of the service to the target server;
receiving a response time, percentage of CPU usage, and free RAM of the target server;
increasing the number of jobs assigned to the service on the target server in specified intervals to reach an amount of work expected from the target server;
calculating expected response time for the target server based on a single thread single job process for the number of jobs increased;
increasing parallel processes for the target server;
calculating expected response time from the increase in parallel processes; and
discontinuing the increasing when the response time varies more than a specified percentage from the expected response time.
6. The system of claim 5 , wherein the threshold includes a maximum number of services and threads to be assigned.
7. The system of claim 5 , further comprising registering servers having the same services to be run in parallel.
8. The system of claim 5 , wherein responsive to assessment of the capability of other servers, the method includes redistributing work among the other servers as a function of a target time of completion for work running on the servers.
9. A computer program product embodied on a non-transitory computer readable medium, the computer program product including instructions that, when executed by a computer, causes the computer to perform operations comprising:
receiving a request to execute a service;
assessing capacity of a target server to execute the service by at least calibrating the target server with respect to the service with parallel threads as a function of CPU consumption and RAM consumption;
setting a threshold for a number of services to be executed on the target server;
assigning a unique identifier of the service to the target server;
based on the threshold and an assessment of a capability of other servers, assigning at least a portion of the service to the target server;
receiving a response time, percentage of CPU usage, and free RAM of the target server;
increasing the number of jobs assigned to the service on the target server in specified intervals to reach an amount of work expected from the target server;
calculating expected response time for the target server based on a single thread single job process for the number of jobs increased;
increasing parallel processes for the target server;
calculating expected response time from the increase in parallel processes; and
discontinuing the increasing when the response time varies more than a specified percentage from the expected response time.
10. The computer program product of claim 9 , wherein the threshold includes a maximum number of services and threads to be assigned.
11. The computer program product of claim 9 , wherein the operations further comprise registering servers having the same services to be run in parallel.