IP Library Granted Patent US 11,360,881
Granted Patent B2
US 11,360,881 · App. 17/028,565 · Granted Jun 14, 2022

Customizing computer performance tests

Inventors: Richard Gardner (Leesburg, VA); Clayton Myers (Oak Hill, VA); Andrew Smith (Oakton, VA); Timothy Lang (McLean, VA)
Assignee: MicroStrategy Incorporated
G06F11/3688G06F11/3684G06F11/3692
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,360,881
App. No.
17/028,565
Granted
Jun 14, 2022
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for testing performance of server systems. In some implementations, a method includes identifying, by one or more computers, a resource provided by a server system. The method includes generating, by the one or more computers, a performance test for the server system. The performance test includes a series of operations that includes (i) providing the resource and (ii) responding to one or more interactions with the resource. The method includes performing one or more performance testing cycles that each include: causing the server system to perform the series of operations of the performance test; and evaluating one or more performance measures that characterize the performance of the series of operations by the server system.

Claims (73)

1. A method performed by one or more computers, the method comprising:

identifying, by the one or more computers, a resource provided by a server system;

determining, by the one or more computers, one or more interactions to be tested for the resource, the one or more interactions being determined based on user input or records of prior user interactions;

generating, by the one or more computers, a performance test for the server system, the performance test comprising a series of operations for the server system to perform including (i) providing the resource and (ii) responding to the one or more interactions with the resource;

performing one or more performance testing cycles that each include:

causing the server system to perform the series of operations of the performance test; and

evaluating one or more performance measures that characterize the performance of the series of operations by the server system; and

based on the evaluation of the one or more performance measures, providing a notification regarding the operation of the server system.

2. The method of claim 1 , wherein the resource is a first resource and the performance test is a first performance test; and

wherein the method comprises generating one or more other performance tests each involving a respective resource that is different from the first resource, the one or more other performance tests each comprising a respective series of operations that includes (i) providing the respective resource and (ii) responding to one or more interactions with the respective resource; and

wherein performing the one or more performance testing cycles comprises performing, in each of the performance testing cycles, the first performance test and each of the one or more other performance tests.

3. The method of claim 1 , wherein performing the one or more performance testing cycles comprises performing multiple performance testing cycles repeated at an interval.

4. The method of claim 1 , wherein the one or more interactions comprise one or more simulated user interactions with the resource.

5. The method of claim 4 , wherein the resource is a document, and wherein the one or more simulated user interactions comprise an interaction by at least one of navigating to a particular portion of the document, scrolling, editing, annotating, saving, filtering, selecting, highlighting, printing, copying, zooming, clicking, or dragging.

6. The method of claim 4 , wherein the one or more simulated user interactions comprises a sequence of multiple simulated user interactions.

7. The method of claim 1 , comprising, in each of the one or more performance testing cycles, determining one or more performance measures for the series of operations of the performance test, the one or more performance measures being based on:

a duration of time to complete the series of operations;

a duration of time to complete one or more individual operations in the series of operations;

an amount or type of errors occurring during the series of operations; or

an accuracy of a result of performing the series of operations.

8. The method of claim 1 , comprising receiving data indicating a series of user interactions with the resource that a user designated to be included in the performance test; and

wherein the performance test is generated to include responding to the series of user interactions with the resource.

9. The method of claim 1 , wherein generating the performance test comprises:

accessing user activity data indicating multiple sequences of user interactions with the resource over a period of time;

selecting a subset of the user interactions with the resource based on an amount of times the respective user interactions with the resource occur in the multiple sequences of user interactions; and

including, in the series of operations of the performance test, operations representing the interactions in the selected subset.

10. The method of claim 1 , comprising:

receiving activity data indicating user interactions with multiple resources provided by the server system;

evaluating the activity data to identify resources that satisfy one or more criteria; and

providing, to a user device for presentation, a recommendation of one or more identified resources to be tested during the one or more performance testing cycles.

11. The method of claim 1 , comprising:

receiving activity data indicating user interactions with the resource;

evaluating the activity data to identify a sequence of multiple user interactions that was performed at least a threshold number of times; and

providing, to a user device for presentation, a recommendation of the sequence of multiple user interactions to be tested in the performance test.

12. The method of claim 1 , wherein the performance test includes operations that simulate loading of a specific document, presenting a specific portion of the specific document, and activating one or more interactive user interface controls available for the specific portion of the specific document.

13. The method of claim 1 , comprising, based on the evaluation of the one or more performance measures, adjusting one or more operating parameters of the server system.

14. A system comprising:

one or more computers; and

one or more computer-readable media storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:

identifying, by the one or more computers, a resource provided by a server system;

determining, by the one or more computers, one or more interactions to be tested for the resource, the one or more interactions being determined based on user input or records of prior user interactions;

generating, by the one or more computers, a performance test for the server system, the performance test comprising a series of operations for the server system to perform including (i) providing the resource and (ii) responding to the one or more interactions with the resource;

performing one or more performance testing cycles that each include:

causing the server system to perform the series of operations of the performance test; and

evaluating one or more performance measures that characterize the performance of the series of operations by the server system; and

based on the evaluation of the one or more performance measures, providing a notification regarding the operation of the server system.

15. The system of claim 14 , wherein the one or more interactions comprise one or more simulated user interactions with the resource.

16. The system of claim 14 , wherein the operations comprise:

receiving activity data indicating user interactions with multiple resources provided by the server system;

evaluating the activity data to identify resources that satisfy one or more criteria; and

providing, to a user device for presentation, a recommendation of one or more identified resources to be tested during the one or more performance testing cycles.

17. The system of claim 14 , wherein the operations comprise:

receiving activity data indicating user interactions with the resource;

evaluating the activity data to identify a sequence of multiple user interactions that was performed at least a threshold number of times; and

providing, to a user device for presentation, a recommendation of the sequence of multiple user interactions to be tested in the performance test.

18. The system of claim 14 , wherein the performance test includes operations that simulate loading of a specific document, presenting a specific portion of the specific document, and activating one or more interactive user interface controls available for the specific portion of the specific document.

19. One or more non-transitory computer-readable media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

identifying, by the one or more computers, a resource provided by a server system;

determining, by the one or more computers, one or more interactions to be tested for the resource, the one or more interactions being determined based on user input or records of prior user interactions;

generating, by the one or more computers, a performance test for the server system, the performance test comprising a series of operations for the server system to perform including (i) providing the resource and (ii) responding to the one or more interactions with the resource;

performing one or more performance testing cycles that each include:

causing the server system to perform the series of operations of the performance test; and

evaluating one or more performance measures that characterize the performance of the series of operations by the server system; and

based on the evaluation of the one or more performance measures, adjusting one or more configuration settings of the server system.

20. A method performed by one or more computers, the method comprising:

identifying, by the one or more computers, a resource provided by a server system;

determining, by the one or more computers, one or more interactions to be tested for the resource, the one or more interactions being determined based on user input or records of prior user interactions;

generating, by the one or more computers, a performance test for the server system, the performance test comprising a series of operations for the server system to perform including (i) providing the resource and (ii) responding to the one or more interactions with the resource;

performing one or more performance testing cycles that each include:

causing the server system to perform the series of operations of the performance test; and

evaluating one or more performance measures that characterize the performance of the series of operations by the server system; and

based on the evaluation of the one or more performance measures, adjusting one or more configuration settings of the server system.

21. The method of claim 20 , wherein adjusting the one or more configuration settings of the server system based on the evaluation of the one or more performance measures comprises: based on the evaluation indicating a decrease in performance of the server system, allocating additional hardware resources to the server system.

Assignments (5)
CHANGE OF NAME Recorded Sep 19, 2025
From: MICROSTRATEGY INCORPORATED
To: STRATEGY INC
Reel/Frame 072910/0511 →
TERMINATION AND RELEASE OF PATENT SECURITY AGREEMENT AT REEL/FRAME: 056647/0687, REEL/FRAME: 057435/0023, REEL/FRAME: 059256/0247, REEL/FRAME: 062794/0255 AND REEL/FRAME: 066663/0713 Recorded Sep 26, 2024
From: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS SUCCESSOR IN INTEREST TO U.S. BANK NATIONAL ASSOCIATION, IN ITS CAPACITY AS COLLATERAL AGENT FOR THE SECURED PARTIES
To: MICROSTRATEGY INCORPORATED; MICROSTRATEGY SERVICES CORPORATION
Reel/Frame 069065/0539 →
SUPPLEMENTARY PATENT SECURITY AGREEMENT Recorded Feb 17, 2023
From: MICROSTRATEGY INCORPORATED; MICROSTRATEGY SERVICES CORPORATION
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS SUCCESSOR IN INTEREST TO U.S. BANK NATIONAL ASSOCIATION, IN ITS CAPACITY AS COLLATERAL AGENT FOR THE SECURED PARTIES
Reel/Frame 062794/0255 →
SECURITY INTEREST Recorded Jun 22, 2021
From: MICROSTRATEGY INCORPORATED
To: U.S. BANK NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 056647/0687 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 23, 2020
From: GARDNER, RICHARD; MYERS, CLAYTON; SMITH, ANDREW; LANG, TIMOTHY
To: MICROSTRATEGY INCORPORATED
Reel/Frame 053854/0618 →