IP Library Granted Patent US 11,327,952
Granted Patent B2
US 11,327,952 · App. 16/563,204 · Granted May 10, 2022

Multivariate auto-differential correlation engine in a computing environment

Inventors: Ajay Krishna Borra (Hyderabad, IN); Gokulakrishnan Gopalakrishnan (Hyderabad, IN); Manpreet Singh (Hyderabad, IN); Brian Toal (San Francisco, CA); Laksh Venka (San Francisco, CA); Metarya Ruparel (Hyderabad, IN)
Assignee: salesforce.com, Inc.
G06F16/2365G06F16/2379
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,327,952
App. No.
16/563,204
Granted
May 10, 2022
Kind
B2
Abstract

A metric data stream for a plurality of metrics may be retrieved from a database system. Each metric may measure a respective computing characteristic. The metric data stream may include a plurality of values for each of a sequence of time intervals. Each value may correspond with a respective one of the metrics. A plurality of metric correlation matrices may be determined for the metrics, each of which is associated with a respective time period in the metric data stream. A subset of comparison metric correlation matrices may be selected from the plurality of metric correlation metric matrices. A designated anomaly score may be determined for a designated time period by comparing a designated metric correlation matrix associated with the designated time period with the selected subset of comparison metric correlation metric matrices.

Claims (41)

1. A computer-implemented method implemented in a database system within an on-demand computing services environment, the method comprising:

retrieving from the database system a metric data stream for a plurality of metrics, each metric measuring a respective computing characteristic, the metric data stream including a plurality of values for each of a sequence of time intervals, each value corresponding with a respective one of the metrics;

determining, using one or more processors, a plurality of metric correlation matrices for the metrics, each metric correlation matrix associated with a respective time period in the metric data stream, each time period including a respective plurality of the time intervals;

selecting, using one or more processors, a subset of comparison metric correlation matrices from the plurality of metric correlation metric matrices;

determining via a processor a plurality of anomaly scores each corresponding with a respective time period, the plurality of anomaly scores including a designated anomaly score for a designated time period determined by comparing a designated metric correlation matrix associated with the designated time period with the selected subset of comparison metric correlation metric matrices;

determining a plurality of moving average anomaly scores, each moving average anomaly score representing a respective average of the anomaly scores associated with a respective time window; and

when it is determined that the designated anomaly score exceeds a designated threshold, storing the designated anomaly score in the database system.

2. The computer-implemented method recited in claim 1 , the method further comprising:

performing seasonality detection on the metric stream to identify a plurality of season periods, one or more of the metric values varying systematically with the season periods.

3. The computer-implemented method recited in claim 2 , the method further comprising:

determining a respective annotation value for each of the identified season periods via dynamic historic window annotation, the annotation value being selected from a set of season period classification values.

4. The computer-implemented method recited in claim 3 , wherein each of the subset of comparison metric correlation matrices corresponds to a respective one of the identified season periods.

5. The computer-implemented method recited in claim 4 , wherein the subset of metric correlation matrices are selected based on the annotation values.

6. The computer-implemented method recited in claim 2 , wherein performing seasonality detection comprises applying seasonal and trend decomposition using Loess.

7. The computer-implemented method recited in claim 1 , wherein each of the metrics corresponds to a respective value measured at a computing system.

8. The computer-implemented method recited in claim 7 , wherein the computing system represents a portion of the on-demand computing services environment, the on-demand computing services environment configured to provide computing services to a plurality of service recipients via the internet.

9. The computer-implemented method recited in claim 1 , wherein the database system includes a multitenant database storing information associated with a plurality of tenants.

10. The computer-implemented method recited in claim 1 , wherein the database system includes a dynamic-schema database in which database column attributes are dynamically determined based on object relational mapping.

11. A computing system within an on-demand computing services environment, the computing system configured to perform a method comprising:

retrieving from the database system a metric data stream for a plurality of metrics, each metric measuring a respective computing characteristic, the metric data stream including a plurality of values for each of a sequence of time intervals, each value corresponding with a respective one of the metrics;

determining, using one or more processors, a plurality of metric correlation matrices for the metrics, each metric correlation matrix associated with a respective time period in the metric data stream, each time period including a respective plurality of the time intervals;

selecting, using one or more processors, a subset of comparison metric correlation matrices from the plurality of metric correlation metric matrices;

determining via a processor a plurality of anomaly scores each corresponding with a respective time period, the plurality of anomaly scores including a designated anomaly score for a designated time period determined by comparing a designated metric correlation matrix associated with the designated time period with the selected subset of comparison metric correlation metric matrices;

determining a plurality of moving average anomaly scores, each moving average anomaly score representing a respective average of the anomaly scores associated with a respective time window; and

when it is determined that the designated anomaly score exceeds a designated threshold, storing the designated anomaly score in the database system.

12. The computing system recited in claim 11 , the method further comprising:

performing seasonality detection on the metric stream to identify a plurality of season periods, one or more of the metric values varying systematically with the season periods.

13. The computing system recited in claim 12 , the method further comprising:

determining a respective annotation value for each of the identified season periods via dynamic historic window annotation, the annotation value being selected from a set of season period classification values.

14. The computing system recited in claim 13 , wherein each of the subset of comparison metric correlation matrices corresponds to a respective one of the identified season periods.

15. The computing system recited in claim 14 , wherein the subset of metric correlation matrices are selected based on the annotation values.

16. The computing system recited in claim 12 , wherein performing seasonality detection comprises applying seasonal and trend decomposition using Loess.

17. The computing system recited in claim 11 , wherein each of the metrics corresponds to a respective value measured at a computing system.

18. The computing system recited in claim 17 , wherein the computing system represents a portion of an on-demand computing services environment configured to provide computing services to a plurality of service recipients via the internet.

19. One or more non-transitory computer readable media having instructions stored thereon for performing a computer-implemented method implemented in a database system within an on-demand computing services environment, the method comprising:

retrieving from the database system a metric data stream for a plurality of metrics, each metric measuring a respective computing characteristic, the metric data stream including a plurality of values for each of a sequence of time intervals, each value corresponding with a respective one of the metrics;

determining, using one or more processors, a plurality of metric correlation matrices for the metrics, each metric correlation matrix associated with a respective time period in the metric data stream, each time period including a respective plurality of the time intervals;

selecting, using one or more processors, a subset of comparison metric correlation matrices from the plurality of metric correlation metric matrices;

determining via a processor a plurality of anomaly scores each corresponding with a respective time period, the plurality of anomaly scores including a designated anomaly score for a designated time period determined by comparing a designated metric correlation matrix associated with the designated time period with the selected subset of comparison metric correlation metric matrices;

determining a plurality of moving average anomaly scores, each moving average anomaly score representing a respective average of the anomaly scores associated with a respective time window; and

when it is determined that the designated anomaly score exceeds a designated threshold, storing the designated anomaly score in the database system.

Assignments (2)
CHANGE OF NAME Recorded Dec 18, 2024
From: SALESFORCE.COM, INC.
To: SALESFORCE, INC.
Reel/Frame 069717/0444 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 6, 2019
From: BORRA, AJAY KRISHNA; GOPALAKRISHNAN, GOKULAKRISHNAN; SINGH, MANPREET; TOAL, BRIAN; VENKA, LAKSH; RUPAREL, METARYA
To: SALESFORCE.COM, INC.
Reel/Frame 050298/0001 →
Continuity (1)
Related Publication 20210073200A1 · Mar 11, 2021
Cited By (1)
US 12,292,816