IP Library Granted Patent US 10,346,295
Granted Patent B2
US 10,346,295 · App. 15/487,727 · Granted Jul 9, 2019

Traffic replay to detect interface changes

Inventors: Harold E. Beyel, III (Bellevue, WA); Piotr Aleksander Ciszewski (Bothell, WA); Daulet Zhanguzin (Bellevue, WA); Syed Wali Hamza (Redmond, WA); Dongyang Cheng (Bellevue, WA)
Assignee: Microsoft Technology Licensing, LLC
G06F11/3692G06F11/3688
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,346,295
App. No.
15/487,727
Granted
Jul 9, 2019
Kind
B2
Abstract

Systems, methods, apparatuses, and software for software testing systems in computing environments are provided herein. In one example, software modules are instantiated within an enveloping framework for replaying or recording of user activity with regards to the software modules. Based at least on the enveloping framework placed in a replay mode of operation, the enveloping framework provides replayed inputs to the software modules based on previously recorded user activity, the replayed inputs each associated with unique identifiers mapping into the previously recorded user activity. The enveloping framework monitors outputs from the software modules and correlates the outputs to the inputs using the unique identifiers. Changes are detected in operation of the software modules based in part on a comparison of previously recorded outputs from the software modules to the outputs that are determined by the software modules when using the replayed inputs.

Claims (67)

1. A computing apparatus comprising:

one or more computer readable storage media;

a processing system comprising a processor operatively coupled with the one or more computer readable storage media; and

program instructions stored on the one or more computer readable storage media, that when executed by the processing system, direct the processing system to at least:

instantiate at least classes associated with one or more software modules within an enveloping framework that forms an encapsulation border for interception of inputs to the one or more software modules and outputs from the one or more software modules;

the enveloping framework configured to:

provide the inputs to the one or more software modules based on previously recorded user activity, the inputs associated with request identifiers mapping into the previously recorded user activity;

intercept the outputs comprising calls made by the one or more software modules to external dependencies by preventing the calls from reaching the external dependencies;

present results to the one or more software modules responsive to the calls to the external dependencies, wherein the results are selected based at least on correlations with the request identifiers associated with the inputs; and

detect changes in operation of the one or more software modules based in part on a comparison of previously recorded outputs from the one or more software modules to replayed outputs that are determined by the one or more software modules when using the inputs and the results.

2. The computing apparatus of claim 1 , wherein the one or more software modules comprise application programming interface (API) software methods contained in one or more classes, and wherein instantiating at least the one or more classes associated with the one or more software modules within the enveloping framework comprises wrapping the one or more classes in the enveloping framework to selectively record or replay boundary activity for the API software methods comprising the inputs and the outputs.

3. The computing apparatus of claim 1 , comprising further program instructions, when executed by the processing system, direct the processing system to at least:

based at least on the enveloping framework configured to record the user activity to the one or more software modules:

instantiate at least the classes associated with the one or more software modules within the enveloping framework for recording the user activity associated with the one or more software modules;

monitor user inputs among the user activity to the one or more software modules, and store input data related to the user inputs in association with corresponding request identifiers;

monitor outputs determined by the one or more software modules responsive to the user inputs, and store output data related to the outputs in association with ones of the request identifiers that correspond to the user inputs; and

monitor external activity generated responsive to the user inputs between the one or more software modules and the external dependencies, and store dependency data related to the external activity in association with the ones of the request identifiers that correspond to the user inputs.

4. The computing apparatus of claim 3 , comprising further program instructions, when executed by the processing system, direct the processing system to at least:

during recording of the user activity, form mappings between the request identifiers and sets of user activity values comprising the input data, the output data, and the dependency data; and

store the mappings in one or more data structures for replay by the enveloping framework.

5. The computing apparatus of claim 4 , comprising further program instructions, when executed by the processing system, direct the processing system to at least:

during replay of the user activity, initiate a series of requests comprising the input data to the one or more software modules, and track the replayed outputs using the request identifiers associated with the input data of the series of requests for comparison with the output data to detect the changes in the operation of the one or more software modules.

6. The computing apparatus of claim 3 , wherein the enveloping framework determines whether to replay or to record the user activity with regards to the one or more software modules based at least on an instruction issued to the enveloping framework for instantiation of the one or more software modules.

7. The computing apparatus of claim 1 , wherein the comparison of the previously recorded outputs to the replayed outputs comprises correlating, via the request identifiers, the previously recorded outputs to the replayed outputs.

8. The computing apparatus of claim 1 , wherein a previous version of at least one of the software modules was employed to determine the previously recorded user activity; and wherein to detect changes in the operation of the one or more software modules, a comparison of behavior of the one or more software modules is performed using the previously recorded user activity.

9. A method of detecting changes in operation of software elements, comprising:

instantiating at least classes associated with one or more software modules within an enveloping framework that forms an encapsulation border for interception of inputs to the one or more software modules and outputs from to the one or more software modules;

in the enveloping framework, providing the inputs to the one or more software modules based on previously recorded user activity, the inputs associated with request identifiers mapping into the previously recorded user activity;

in the enveloping framework, intercepting the outputs comprising calls made by the one or more software modules to external dependencies by preventing the calls from reaching the external dependencies;

in the enveloping framework, presenting results to the one or more software modules responsive to the calls to the external dependencies, wherein the results are selected based at least on correlations with the request identifiers associated with the inputs; and

detecting changes in operation of the one or more software modules based in part on a comparison of previously recorded outputs from the one or more software modules to replayed outputs that are determined by the one or more software modules when using the inputs and the results.

10. The method of claim 9 , wherein the one or more software modules comprise application programming interface (API) software methods contained in one or more classes, and wherein instantiating at least the classes associated with the one or more software modules within the enveloping framework comprises wrapping the one or more classes in the enveloping framework to selectively record or replay boundary activity for the API software methods comprising the inputs and the outputs.

11. The method of claim 9 , further comprising:

in the enveloping framework, based at least on the enveloping framework configured to record the user activity to the one or more software modules:

instantiating at least the classes associated with the one or more software modules within the enveloping framework for recording the user activity associated with the one or more software modules;

monitoring user inputs among the user activity to the one or more software modules, and storing input data related to the user inputs in association with corresponding request identifiers;

monitoring the outputs determined by the one or more software modules responsive to the user inputs, and storing output data related to the outputs in association with ones of the request identifiers that correspond to the user inputs; and

monitoring external activity generated responsive to the user inputs between the one or more software modules and the external dependencies, and storing dependency data related to the external activity in association with the ones of the request identifiers that correspond to the user inputs.

12. The method of claim 11 , further comprising:

during recording of the user activity, forming mappings between the request identifiers and sets of user activity values comprising the input data, the output data, and the dependency data; and

storing the mappings in one or more data structures for replay by the enveloping framework.

13. The method of claim 12 , further comprising:

during replay of the user activity, initiating a series of requests comprising the input data to the one or more software modules, and tracking the replayed outputs using the request identifiers associated with the input data of the series of requests for comparison with the output data to detect the changes in the operation of the one or more software modules.

14. The method of claim 11 , wherein the enveloping framework determines whether to replay or to record the user activity with regards to the one or more software modules based at least on an instruction issued to the enveloping framework for instantiation of the one or more software modules.

15. The method of claim 9 , wherein the comparison of the previously recorded outputs to the replayed outputs comprises correlating, via the request identifiers, the previously recorded outputs to the replayed outputs.

16. The method of claim 9 , wherein a previous version of at least one of the software modules was employed to determine the previously recorded user activity; and wherein to detect changes in the operation of the one or more software modules, a comparison of behavior of the one or more software modules is performed using the previously recorded user activity.

17. A computing apparatus comprising:

one or more computer readable storage media;

a processing system comprising a processor operatively coupled with the one or more computer readable storage media; and

program instructions stored on the one or more computer readable storage media, that when executed by the processing system, direct the processing system to at least:

instantiate at least classes associated with one or more software modules within an enveloping framework that forms a boundary for interception of inputs to the one or more software modules and interception of outputs from the one or more software modules based at least on an instruction issued to the enveloping framework for instantiation of the one or more software modules;

based at least on the enveloping framework placed in a replay mode of operation:

provide the inputs comprising replayed activity to the one or more software modules based on previously recorded user activity, the inputs each associated with unique identifiers mapping into the previously recorded user activity;

intercept the outputs by preventing calls made by the one or more software modules from reaching external services; and

monitor the outputs from the one or more software modules and correlate the outputs to the inputs using the unique identifiers; and

detect changes in operation of the one or more software modules based in part on a comparison of previously recorded outputs from the one or more software modules to the outputs that are determined by the one or more software modules when using the inputs.

18. The computing apparatus of claim 17 , comprising further program instructions, when executed by the processing system, direct the processing system to at least:

based at least on the enveloping framework placed in a record mode of operation:

instantiate at least the classes associated with the one or more software modules within the enveloping framework for recording the user activity associated with the one or more software modules;

monitor user inputs among the user activity to the one or more software modules, and store input data related to the user inputs in association with corresponding unique identifiers;

monitor outputs determined by the one or more software modules responsive to the user inputs, and store output data related to the outputs in association with ones of the unique identifiers that correspond to the user inputs.

19. The computing apparatus of claim 17 , comprising further program instructions, when executed by the processing system, direct the processing system to at least:

based at least on the enveloping framework placed in the replay mode of operation, intercept calls made by the one or more software modules to external services by withholding the calls to the external services and emulating behavior of the external services by at least presenting results to the one or more software modules responsive to the calls to the external services, wherein the results are selected based at least on correlations with the unique identifiers associated with the replayed inputs.

20. The computing apparatus of claim 17 ,

wherein the enveloping framework determines whether to replay or to record the user activity with regards to the one or more software modules based at least on an instruction issued to the enveloping framework for instantiation of the one or more software modules;

wherein the comparison of the previously recorded outputs to the replayed outputs comprises correlating, via the unique identifiers, the previously recorded outputs to the replayed outputs; and

wherein a previous version of at least one of the software modules was employed to determine the previously recorded user activity; and wherein to detect changes in the operation of the one or more software modules, a comparison of behavior of the one or more software modules is performed using the previously recorded user activity.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 14, 2017
From: BEYEL, HAROLD E., III; CISZEWSKI, PIOTR ALEKSANDER; ZHANGUZIN, DAULET; HAMZA, SYED WALI; CHENG, DONGYANG
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 042009/0823 →
Continuity (1)
Related Publication 20180300228A1 · Oct 18, 2018