IP Library › Granted Patent US 11,256,598
Granted Patent B2
US 11,256,598 · App. 16/818,656 · Granted Feb 22, 2022

Automated selection of performance monitors

Inventors: Saurabh Jha (Urbana, IL); Amos A. Omokpo (Richmond, TX); Karthick Rajamani (Austin, TX); HariGovind Venkatraj Ramasamy (Round Rock, TX)
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
G06F11/3452G06F9/542G06F11/3476G06F17/18G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,256,598
App. No.
16/818,656
Granted
Feb 22, 2022
Kind
B2
Abstract

An embodiment includes extracting statistical data associated with invocation of an application programming interface (API) from a log and using the statistical data to calculate a performance value and generate an aggregate dataset that combines the performance value with performance values associated with other invocations of the API. The embodiment includes calculating metric values for performance values for respective time intervals of a time period and calculating mean and standard deviation values of the metric values for the time period. The embodiment includes selecting the API as a candidate API and detecting a Customer Impacting Event (CIE) by applying a machine learning algorithm using monitored values associated with the candidate API during a time frame defined by a rolling window. The embodiment also includes automatically initiating a selected alert from among a plurality of alert options based at least in part on the monitored values associated with the CIE.

Claims (49)

1. A computer implemented method comprising:

deploying a plurality of application programming interfaces (APIs) in a cloud computing infrastructure;

extracting, by a processor, statistical data from an API call log, the statistical data being associated with an invocation of an API by an application;

calculating, by the processor, a performance value associated with the invocation of the API using the statistical data;

generating, by the processor, an aggregate dataset that combines the performance value associated with the invocation of the API with performance values associated with respective previous invocations of the API;

calculating, by the processor, metric values for respective time intervals of a time period, wherein the metric values are associated with performance values during the respective time intervals;

calculating, by the processor, mean and standard deviation values of the metric values for the time period;

selecting, by the processor, the API as a candidate API, the API being selected from among the plurality of APIs based at least in part on criteria involving performance values of the APIs, wherein the criteria comprise (i) an arrival rate of the candidate API is within a band of highest arrival rates for the plurality of APIs, (ii) a standard deviation of a metric for the candidate API is not greater than a mean of the metric for the API, and (iii) a ratio ρ between a mean arrival rate λ for the candidate API and a mean response time, μ for the candidate API times a number of replicas c for the candidate API according to Expression p=λ/c·μ is less than 1;

detecting, by the processor, a Customer Impacting Event (CIE) by applying a machine learning algorithm using monitored values associated with the candidate API during a time frame defined by a rolling window; and

responsive to the detecting of the CIE, automatically initiating, by the processor, a selected alert from among a plurality of alert options based at least in part on the monitored values associated with the CIE.

2. The computer implemented method of claim 1 , wherein the statistical data includes response time data and arrival rate data.

3. The computer implemented method of claim 1 , wherein the extracting further comprises extracting statistical data from the API call log associated with invocation of a plurality of APIs, the plurality of APIs including said API.

4. The computer implemented method of claim 1 , wherein the calculating of a performance value comprises calculating a response time.

5. The computer implemented method of claim 1 , wherein the calculating of a performance value further comprises calculating an arrival rate.

6. The computer implemented method of claim 1 , wherein a type of metric used for calculating the metric values is selected from the group consisting of a median value, a 95th percentile value, and a 99th percentile value.

7. The computer implemented method of claim 1 , wherein the time intervals include a plurality of equal time intervals.

8. The computer implemented method of claim 1 , wherein the machine learning algorithm provides an indication of monitored values that are outlier values indicative of the CIE.

9. The computer implemented method of claim 1 , wherein the monitored values comprise response time values and arrival rate values.

10. The computer implemented method of claim 1 , wherein the plurality of alert options include an outage alert and a performance alert.

11. A computer program product for automated selection of performance and reliability monitors, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations comprising:

deploying a plurality of application programming interfaces (APIs) in a cloud computing infrastructure;

extracting, by a processor, statistical data from an API call log, the statistical data being associated with an invocation of an API by an application;

calculating, by the processor, a performance value associated with the invocation of the API using the statistical data;

generating, by the processor, an aggregate dataset that combines the performance value associated with the invocation of the API with performance values associated with respective previous invocations of the API;

calculating, by the processor, metric values for respective time intervals of a time period, wherein the metric values are associated with performance values during the respective time intervals;

calculating, by the processor, mean and standard deviation values of the metric values for the time period;

selecting, by the processor, the API as a candidate API, the API being selected from among the plurality of APIs based at least in part on criteria involving performance values of the APIs, wherein the criteria comprise (i) an arrival rate of the candidate API is within a band of highest arrival rates for the plurality of APIs, (ii) a standard deviation of a metric for the candidate API is not greater than a mean of the metric for the API, and (iii) a ratio ρ between a mean arrival rate λ for the candidate API and a mean response time μ for the candidate API times a number of replicas c for the candidate API according to Expression p=λ/c·μ is less than 1;

detecting, by the processor, a CIE by applying a machine learning algorithm using monitored values associated with the candidate API during a time frame defined by a rolling window; and

responsive to the detecting of the CIE, automatically initiating, by the processor, a selected alert from among a plurality of alert options based at least in part on the monitored values associated with the CIE.

12. The computer program product of claim 11 , wherein the stored program instructions are stored in a computer readable storage device in a data processing system, and wherein the stored program instructions are transferred over a network from a remote data processing system.

13. The computer program product of claim 11 , wherein the stored program instructions are stored in a computer readable storage device in a server data processing system, and wherein the stored program instructions are downloaded over a network to a remote data processing system for use in a computer readable storage device associated with the remote data processing system, further comprising:

metering use of the computer usable code associated with the request; and

generating an invoice based on the metered use.

14. The computer program product of claim 11 , wherein the statistical data includes response time data and arrival rate data.

15. The computer program product of claim 11 , wherein a type of metric used for calculating the metric values is selected from the group consisting of a median value, a 95th percentile value, and a 99th percentile value.

16. The computer program product of claim 11 , wherein the plurality of alert options include an outage alert and a performance alert.

17. A computer system comprising a processor and one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable by the processor to cause the processor to perform operations comprising:

deploying a plurality of application programming interfaces (APIs) in a cloud computing infrastructure;

extracting, by a processor, statistical data from an API call log, the statistical data being associated with an invocation of an API by an application;

calculating, by the processor, a performance value associated with the invocation of the API using the statistical data;

generating, by the processor, an aggregate dataset that combines the performance value associated with the invocation of the API with performance values associated with respective previous invocations of the API;

calculating, by the processor, metric values for respective time intervals of a time period, wherein the metric values are associated with performance values during the respective time intervals;

calculating, by the processor, mean and standard deviation values of the metric values for the time period;

selecting, by the processor, the API as a candidate API, the API being selected from among the plurality of APIs based at least in part on criteria involving performance values of the APIs, wherein the criteria comprise (i) an arrival rate of the candidate API is within a band of highest arrival rates for the plurality of APIs, (ii) a standard deviation of a metric for the candidate API is not greater than a mean of the metric for the API, and (iii) a ratio ρ between a mean arrival rate λ for the candidate API and a mean response time μ for the candidate API times a number of replicas c for the candidate API according to Expression p=λ/c·μ is less than 1;

detecting, by the processor, a CIE by applying a machine learning algorithm using monitored values associated with the candidate API during a time frame defined by a rolling window; and

responsive to the detecting of the CIE, automatically initiating, by the processor, a selected alert from among a plurality of alert options based at least in part on the monitored values associated with the CIE.

18. The computer system of claim 17 , wherein the statistical data includes response time data and arrival rate data.

19. The computer system of claim 17 , wherein a type of metric used for calculating the metric values is selected from the group consisting of a median value, a 95th percentile value, and a 99th percentile value.

20. The computer system of claim 17 , wherein the plurality of alert options include an outage alert and a performance alert.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 13, 2020
From: JHA, SAURABH; OMOKPO, AMOS A.; RAJAMANI, KARTHICK; RAMASAMY, HARIGOVIND VENKATRAJ
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 052112/0209 →
Continuity (1)
Related Publication 20210286699A1 · Sep 16, 2021
Cited By (1)
US 12,572,396