Granted Patent B1

US 12,645,563 · App. 19/201,770 · Granted Jun 2, 2026

Inventor: Leonid Kuperman (North York, CA)

Assignee: Cast AI Group Inc.

G06F11/3612G06F16/284G06F11/3419H04L41/5054H04L43/0817H04L51/214H04L63/029

Abstract

Claims (80)

1 . A method for autonomous application performance optimization, the method comprising:

collecting production telemetry data about an application in a production environment, the telemetry data including one or more of a latency, CPU utilization, memory utilization, throughput, or error rate;

analyzing the production telemetry data to detect performance inefficiencies of the application in the production environment,

wherein the production telemetry data is analyzed using one or more large language models;

identifying a first intervention based on the detected performance inefficiencies,

wherein identifying the first intervention comprises modifying code of the application using the one or more large language models to modify a parameter of the application;

deploying the first intervention onto an application in a sandbox environment for validation;

collecting sandbox telemetry data about the application with the first intervention applied in the sandbox environment;

analyzing the sandbox telemetry data to assess impact of the first intervention in the sandbox environment;

determining whether performance of the application with the first intervention has improved by at least a predetermined threshold, the predetermined threshold associated with at least one of: a latency reduction, a CPU utilization reduction, a memory utilization reduction, a throughput increase, or an error rate reduction;

in response to determining that the performance of the application with the first intervention has improved by at least a predetermined threshold, deploying the first intervention to the application in the production environment;

in response to determining that the performance of the application with the first intervention has not improved by the predetermined threshold,

identifying a second intervention based on the detected performance inefficiencies of the application without the first intervention and the assessed impact of the first intervention; and

deploying the second intervention onto the application in the sandbox environment for validation.

2 . The method of claim 1 , wherein collecting production telemetry data comprises:

deploying agents onto Kubernetes clusters in the production environment configured to collect one or more of logs, metrics, and traces of corresponding Kubernetes clusters.

3 . The method of claim 1 , wherein analyzing the telemetry data comprises identifying anomalous patterns using a machine learning model trained on historical performance data.

4 . The method of claim 1 , wherein sandbox environment comprises a container that mirrors the production environment, and the application is deployed to the container and the first intervention is applied to the application within the sandbox environment without impacting workloads in the production environment.

5 . The method of claim 1 , wherein the intervention comprises selecting a large language model (LLM) from a plurality of LLMs based on one or more of response latency, cost-per token, or GPU resource utilization associated with each of the plurality of LLMs.

6 . The method of claim 5 , wherein the intervention comprises:

selecting a geographic region from a plurality of geographic regions that offer compute instances with GPU resources;

provisioning a compute instance with GPU resources within the selected geographic region; and

deploying the selected LLM onto the provisioned compute instance.

7 . The method of claim 6 , wherein the geographic region is selected based on one or more of: GPU availability, current GPU utilization rates, historical failure rates, eviction trends, network latency, provisioning latency, or spot instance interruption rates.

8 . The method of claim 1 , wherein the intervention comprises applying security patches.

9 . The method of claim 8 , wherein applying security patches comprises:

parsing infrastructure-as-code (IaC) configuration files associated with Kubernetes clusters;

identifying security misconfigurations within the parsed IaC configuration files; and

generating code changes that modify lines of code in the IaC configuration files.

10 . The method of claim 1 , further comprising:

receiving comments, from a client device of a user, on a first pull request;

generating a third intervention based on the comments on the first pull request; and

generating a second pull request based on the third intervention.

11 . The method of claim 1 , wherein the intervention comprises:

identifying a previously deployed version of the application with stable performance metrics; and

initiating a rollback operation to restore the application to the previous deployed version.

12 . The method of claim 1 , wherein the intervention comprises refactoring a recursive function into an iterative implementation to reduce latency.

13 . The method of claim 1 , wherein the intervention further comprises modifying a configuration parameter of the application, the configuration parameter including at least one of: CPU resource limits, memory allocation, or database query caching.

14 . The method of claim 1 , wherein the predetermined threshold comprises at least a predetermined percentage of reduction in latency or compute resource consumption.

15 . A non-transitory computer readable storage medium having instructions encoded thereon that, when executed by one or more processors, cause the one or more processors to perform steps comprising:

collecting production telemetry data about an application in a production environment;

analyzing the production telemetry data to detect performance inefficiencies of the application in the production environment,

wherein the production telemetry data is analyzed using one or more machine learning models;

generating, using the one or more machine learning models, a first intervention comprising a code refactor to address at least one detected performance inefficiency,

wherein generating the first intervention comprises modifying code of the application using the one or more machine learning models based on analyzing the production telemetry data;

deploying the first intervention onto an application in a sandbox environment for validation;

collecting sandbox telemetry data about the application with the first intervention applied in the sandbox environment;

analyzing the sandbox telemetry data to assess impact of the first intervention in the sandbox environment;

determining whether performance of the application with the first intervention has improved by at least a predetermined threshold, compared to previous performance in the production environment;

in response to determining that the performance of the application with the first intervention has improved by at least a predetermined threshold, deploying the first intervention to the application in the production environment;

in response to determining that the performance of the application with the first intervention has not improved by the predetermined threshold,

identifying a second intervention based on the detected performance inefficiencies of the application without the first intervention and the assessed impact of the first intervention; and

deploying the second intervention onto the application in the sandbox environment for validation.

16 . The non-transitory computer readable storage medium of claim 15 , wherein collecting production telemetry data comprises:

deploying agents onto Kubernetes clusters in the production environment configured to collect one or more of logs, metrics, and traces of corresponding Kubernetes clusters.

17 . The non-transitory computer readable storage medium of claim 15 , wherein analyzing the telemetry data comprises identifying anomalous patterns using a machine learning model trained on historical performance data.

18 . The non-transitory computer readable storage medium of claim 15 , wherein sandbox environment comprises a container that mirrors the production environment, and the application is deployed to the container and the first intervention is applied to the application within the sandbox environment without impacting workloads in the production environment.

19 . The non-transitory computer readable storage medium of claim 15 , wherein the intervention comprises selecting a large language model (LLM) from a plurality of LLMs based on one or more of response latency, cost-per token, or GPU resource utilization associated with each of the plurality of LLMs.

20 . The non-transitory computer readable storage medium of claim 19 , wherein the intervention comprises:

selecting a geographic region from a plurality of geographic regions that offer compute instances with GPU resources;

provisioning a compute instance with GPU resources within the selected geographic region; and

deploying the selected LLM onto the provisioned compute instance.

21 . The non-transitory computer readable storage medium of claim 20 , wherein the geographic region is selected based on one or more of: GPU availability, current GPU utilization rates, historical failure rates, eviction trends, network latency, provisioning latency, or spot instance interruption rates.

22 . The non-transitory computer readable storage medium of claim 15 , wherein the intervention comprises applying security patches.

23 . A computing system, comprising:

one or more processors; and

a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by one or more processors, cause the one or more processors to perform steps comprising:

collecting production telemetry data about an application in a production environment, the telemetry data including one or more of a latency, CPU utilization, memory utilization, throughput, or error rate;

analyzing the production telemetry data to detect performance inefficiencies of the application in the production environment,

wherein the production telemetry data is analyzed using one or more machine learning models;

identifying a first intervention based on the detected performance inefficiencies,

wherein identifying the first intervention comprises modifying code of the application using the one or more machine learning models based on analyzing the production telemetry data;

deploying the first intervention onto an application in a sandbox environment for validation;

collecting sandbox telemetry data about the application with the first intervention applied in the sandbox environment;

analyzing the sandbox telemetry data to assess impact of the first intervention in the sandbox environment;

determining whether performance of the application with the first intervention has improved by at least a predetermined threshold, the predetermined threshold associated with at least one of: a latency reduction, a CPU utilization reduction, a memory utilization reduction, a throughput increase, or an error rate reduction;

in response to determining that the performance of the application with the first intervention has improved by at least a predetermined threshold, deploying the first intervention to the application in the production environment;

in response to determining that the performance of the application with the first intervention has not improved by the predetermined threshold,

identifying a second intervention based on the detected performance inefficiencies of the application without the first intervention and the assessed impact of the first intervention; and

deploying the second intervention onto the application in the sandbox environment for validation.

Assignments (1)

Continuity (1)

References Cited (31)