Computer software architecture for execution efficiency
The present disclosure pertains to an improved computing architecture allowing for better utilization of CPU, memory, network bandwidth, and other computing resources, particularly within the context of a decision service system that handles client requests. These methods and systems may be directed to the evaluation and use of a partial response from a decision service system when the generation of a full response by the decision service system is predicted to take longer than what is expected.
1. A method, comprising:
receiving, from a client device and at a decision service system, a request for a response to a decision service query, the response associated with a threshold service level agreement (SLA) time for the decision service system to respond to the client device with a full response of the response after the receiving the request;
predicting that a first total computation time for a plurality of nodes of the decision service system to execute a plurality of tasks, respectively, exceeds the threshold SLA time, wherein a first execution of the plurality of tasks by the plurality of nodes enables the response to be generated;
determining a quality level of a partial response of the response that includes results of a second execution of one or more first tasks of the plurality of tasks by one or more first nodes of the plurality of nodes, respectively, wherein a duration for completing the second execution is less than the threshold SLA time and a number of the one or more first nodes is less than a number of the plurality of nodes; and
responding, by the decision service system, to the client device with the partial response based on whether the quality level of the partial response exceeds a threshold quality level.
2. The method of claim 1 , wherein each of the plurality of tasks is a pre-defined task assigned to one of the plurality of nodes, wherein the predicting comprises dynamically computing, using a machine learning model, a threshold time for the one of the plurality of nodes to execute the pre-defined task.
3. The method of claim 2 , wherein the machine learning model includes neural network machine learning model.
4. The method of claim 1 , wherein the plurality of tasks includes a data gathering task collecting data required for generating the response, a data validation task validating the gathered data, a computation task performing computations using the validated data, and a response building task generating the response based on the computations, and wherein the plurality of tasks are respectively assigned to the plurality of nodes.
5. The method of claim 1 , wherein the partial response includes a callback time stamp indicating a time for the response to be available from the decision service system.
6. The method of claim 5 , wherein the partial response includes a link configured for use by the client device to access the response at or after the time indicated by the callback time stamp.
7. The method of claim 1 , wherein the predicting comprises predicting a second total computation time for one or more second nodes of the one or more first nodes to execute one or more second tasks of the one or more first tasks, respectively, and wherein a third execution of one or more second tasks by the one or more second nodes generates the partial response.
8. A system comprising:
a processor; and
a non-transitory computer-readable medium having stored thereon instructions that are executable by the processor to cause the system to perform operations comprising:
receiving, from a client device, a request for a response to a decision service query, the response associated with a threshold service level agreement (SLA) time for responding to the client device with a full response of the response after the receiving the request;
predicting that a first total computation time for a plurality of nodes to execute a plurality of tasks, respectively, exceeds the threshold SLA time, wherein a first execution of the plurality of tasks by the plurality of nodes enables the response to be generated;
determining a quality level of a partial response of the response that includes results of a the plurality of nodes, respectively, wherein a duration for completing the second execution is less than the threshold SLA time and a number of the one or more first nodes is less than a number of the plurality of nodes; and
responding to the client device with the partial response based on whether the quality level of the partial response exceeds a threshold quality level.
9. The system of claim 8 , wherein each of the plurality of tasks is a pre-defined task assigned to one of the plurality of nodes, wherein the predicting comprises dynamically computing, using a machine learning model, a threshold time for the one of the plurality of nodes to execute the pre-defined task.
10. The system of claim 9 , wherein the machine learning model includes a neural network machine learning model.
11. The system of claim 8 , wherein the plurality of tasks includes a data gathering task collecting data required for generating the response, a data validation task validating the gathered data, a computation task performing computations using the validated data, and a response building task generating the response based on the computations, and wherein the plurality of tasks are respectively assigned to the plurality of nodes.
12. The system of claim 8 , wherein the partial response includes a callback time stamp indicating a time for the response to be available.
13. The system of claim 12 , wherein the partial response includes a link configured for use by the client device to access the response at or after the time indicated by the callback time stamp.
14. The system of claim 8 , wherein the predicting comprises predicting a second total computation time for one or more second nodes of the one or more first nodes to execute one or more second tasks of the one or more first tasks, respectively, and wherein a third execution of the one or more second tasks by the one or more second nodes generates the partial response.
15. A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising:
receiving, from a client device and at a decision service system, a request for a response to a decision service query, the response associated with a threshold service level agreement (SLA) time for the decision service system to respond to the client device with a full response of the response after the receiving the request;
predicting that a first total computation time for a plurality of nodes of the decision service system to execute a plurality of tasks, respectively, exceeds the threshold SLA time, wherein a first execution of the plurality of tasks by the plurality of nodes enables the response to be generated;
determining a quality level of a partial response of the response that includes results of a second execution of one or more first tasks of the plurality of tasks by one or more first nodes of the plurality of nodes, respectively, wherein a duration for completing the second execution is less than the threshold SLA time and a number of the one or more first nodes is less than a number of the plurality of nodes; and
responding, by the decision service system, to the client device with the partial response based on whether the quality level of the partial response exceeds a threshold quality level.
16. The non-transitory machine-readable medium of claim 15 , wherein each of the plurality of tasks is a pre-defined task assigned to one of the plurality of nodes, wherein the predicting comprises dynamically computing, using a machine learning model, a threshold time for the one of the plurality of nodes to execute the pre-defined task.
17. The non-transitory machine-readable medium of claim 16 , wherein the machine learning model includes a neural network machine learning model.
18. The non-transitory machine-readable medium of claim 15 , the plurality of tasks includes a data gathering task collecting data required for generating the response, a data validation task validating the gathered data, a computation task performing computations using the validated data, and a response building task generating the response based on the computations, and wherein the plurality of tasks are respectively assigned to the plurality of nodes.
19. The non-transitory machine-readable medium of claim 15 , wherein the partial response includes a callback time stamp indicating a time for the response to be available from the decision service system, and wherein the partial response includes a link configured for use by the client device to access the response at or after the time indicated by the callback time stamp.
20. The non-transitory machine-readable medium of claim 15 , wherein the predicting comprises predicting a second total computation time for one or more second nodes of the one or more first nodes to execute one or more second tasks of the one or more first tasks, respectively, and wherein a third execution of the one or more second tasks by the one or more second nodes generates the partial response.