IP Library Granted Patent US 10,067,850
Granted Patent B2
US 10,067,850 · App. 15/012,839 · Granted Sep 4, 2018

Load test charts with standard deviation and percentile statistics

Inventors: Charles A. Vazac (San Francisco, CA); Tal Broda (Sunnyvale, CA); Michael Hemmert (Scotts Valley, CA); Mukul Sharma (Mountain View, CA)
Assignee: Akamai Technologies, Inc.
G06F11/3495G06F11/3409G06F11/3414G06F11/3419G06F11/3466G06F17/18H04L67/02G06F2201/875
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,067,850
App. No.
15/012,839
Granted
Sep 4, 2018
Kind
B2
Abstract

A method for real-time analysis of results from a load test performed on a target website includes calculating first-level aggregated test results within each of a plurality of load server instances that generate a load on the target website. The first-level aggregated test results are calculated from data points received by each of the load server instances from the target website. The first-level aggregated test results include a sum of the data points, a count of the number of the data points, a sum of squares of the data points, and an average of the data points. A standard deviation result (STDEV) is calculated and chart is generated on a display via a graphical user interface. The chart provides a visual representation of a performance metric for the load test based on the standard deviation result.

Claims (19)

1. A computer-implemented method for real-time analysis of results obtained from a load test as the load test is being performed on a target website, comprising:

(a) computing first-level aggregated test results within each of a plurality of load server instances that generate a load on the target website, the first-level aggregated test results including first-level statistics computed from a set of data points received over a time period by each of the load server instances from the target website, the first-level statistics being computed by an embedded component coupled with a memory within each of the load server instances, the first-level statistics including a sum of the data points, a count of the number of the data points, a sum of squares of the data points, and an average of the data points;

(b) sending the first-level aggregated test results over a network from each of the load server instances to an associated one of a plurality of analytic server instances;

(c) aggregating, in one or more analytic server instances, all of the first-level aggregated test results computed by the load server instances resulting in second-level aggregated test results;

(d) aggregating in a main instance computer, which includes a processor coupled with a memory, all of the second-level aggregated test results resulting in a cumulative sum, a total count, a total sum of squares, and a cumulative average, of all of the data points across an entire data set;

(e) calculating by the main instance computer a standard deviation result (STDEV) as

STDEV=SQRT[{( X 1 2 +X 2 2 + . . . +X N 2 )+ NA 2 −2 A ( X 1 +X 2 + . . . +X N )}/ N];

where N is the total count, A is the cumulative average, X 1 , X 2 . . . X N are the data points across the entire data set, (X 1 2 +X 2 2 + . . . +X N 2 ) is the total sum of squares, and (X 1 +X 2 + . . . +X N ) is the cumulative sum; and

wherein steps (a)-(e) are each performed in real-time as the load test is being performed on the target website.

2. The computer-implemented method of claim 1 further comprising:

(f) providing on the main instance computer a graphical user interface (GUI) that displays a chart on a screen, the chart providing a visual representation of a performance metric for the load test based on the standard deviation result.

3. The computer-implemented method of claim 2 further comprising:

communicating the standard deviation result to a browser application running on the main instance computer, the browser application providing an analytic dashboard with the graphical user interface.

4. The computer-implemented method of claim 2 wherein the chart is generated in real-time responsive to an input command of the graphical user interface.

5. The computer-implemented method of claim 1 wherein the performance metric comprises a confidence interval percentile (X).

6. The computer-implemented method of claim 1 further comprising calculating by the main instance computer the confidence interval percentile (X) as

X ={( A +( Z *STDEV)};

where Z is a constant used for calculating a certain percentile.

7. The computer-implemented method of claim 1 further comprising calculating by the main instance computer a delta STDEV based on incremental changes occurring as the load test is running in real-time.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 23, 2019
From: VAZAC, CHARLES A.; BRODA, TAL; HEMMERT, MICHAEL; SHARMA, MUKUL
To: SOASTA, INC.
Reel/Frame 048107/0803 →
MERGER Recorded Apr 23, 2018
From: SOASTA, INC.
To: AKAMAI TECHNOLOGIES, INC.
Reel/Frame 045613/0362 →
CORRECTIVE ASSIGNMENT TO CORRECT THE 15338002 RECORDED ON PREVIOUSLY RECORDED ON REEL 044798 FRAME 0558. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNOR (S) HEREBY CONFIRMS THE MERGER. Recorded Apr 2, 2018
From: SOASTA INC.
To: AKAMAI TECHNOLOGIES, INC.
Reel/Frame 046110/0306 →
MERGER Recorded Mar 26, 2018
From: SOASTA, INC.
To: AKAMAI TECHNOLOGIES, INC.
Reel/Frame 045351/0887 →
MERGER Recorded Feb 1, 2018
From: SOASTA, INC.
To: AKAMAI TECHNOLOGIES, INC.
Reel/Frame 044798/0558 →
Continuity (3)
Continuation 13369899 · Feb 9, 2012
Continuation In Part 12804338 · Jul 19, 2010
Related Publication 20160147632A1 · May 26, 2016