Granted Patent B2

US 12,468,623 · App. 18/748,627 · Granted Nov 11, 2025

Inventors: Michael Dale Anderson (Wilton, NY); Byron Asa Wingerd (Cary, NC); Andrea Coombs (East Lansing, MI); Mary Loveless (Arlington, MA)

Assignee: JMP Statistical Discovery LLC

G06F11/3684G06F8/30

Abstract

Claims (91)

1 . A method comprising:

obtaining, by one or more processors, an output dataset generated by a first software program based on an input dataset, wherein the input dataset includes numerical values, wherein the output dataset includes analytical metrics generated based on the numerical values, and wherein the first software program is a first analytical program provided by an entity that is different from an end user;

generating, by the one or more processors, a graphical user interface (GUI) that displays the output dataset to the end user;

detecting, by the one or more processors, a selection by the end user of a particular value of the output dataset in the GUI for use as an expected output value during a subsequent test;

providing, by the one or more processors, a first interface element of the GUI through which the end user can input a maximum number of significant figures for the particular value;

in response to receiving the maximum number of significant figures from the end user via the first interface element, generating, by the one or more processors, a truncated value based on the particular value, wherein the truncated value is a version of the particular value as truncated to the maximum number of significant figures input by the end user;

providing, by the one or more processors, a second interface element showing the truncated value, the second interface element being different from the first interface element;

providing, by the one or more processors, a third interface element through which the end user can input a log relative error value, wherein the third interface element is different than the first and second interface elements;

receiving, by the one or more processors, the log relative error value from the end user via the third interface element;

generating, by the one or more processors, a testing program for testing functionality of a second software program, by inserting the truncated value into testing logic of a predefined code template as the expected output value, wherein the second software program is a second analytical program, and wherein generating the testing program further involves inserting the log relative error value into the testing logic of the predefined code template;

executing, by the one or more processors, the testing program to test the functionality of the second software program and thereby generate test results, wherein the testing program is configured to:

provide the input dataset as input to the second software program;

receive an output from the second software program generated based on the input dataset;

determine whether the output of the second software program deviates from the truncated value by more than a threshold amount; and

in response to determining that the output of the second software program deviates from the truncated value by more than the threshold amount, flag the deviation in the test results; and

outputting, by the one or more processors, the test results to the end user in the GUI.

2 . The method of claim 1 , wherein the first software program is the same as the second software program.

3 . The method of claim 1 , wherein the first software program is a first version of a software program, and the second software program is a second version of the software program.

4 . The method of claim 1 , further comprising:

based on the selection, automatically populating a fourth interface element of the graphical user interface with the particular value, the fourth interface element being editable by the end user, and the fourth interface element being different from the first, second, and third interface elements.

5 . The method of claim 1 , wherein the graphical user interface includes a code preview window that shows source code of the testing program, the source code being editable by the end user in the code preview window to modify the testing program.

6 . The method of claim 1 , further comprising:

selecting the predefined code template from among a plurality of predefined code templates based on a characteristic of the second software program, each code template of the plurality of predefined code templates being configured to test a different software program from the other code templates of the plurality of predefined code templates.

7 . The method of claim 1 , further comprising:

detecting a plurality of selections by the end user of a plurality of locations in the graphical user interface corresponding to a plurality of values of the output dataset; and

based on the plurality of selections, configuring the testing program to include the plurality of values in the testing logic of the predefined code template.

8 . The method of claim 1 , further comprising:

generating the output dataset by applying the first software program to the input dataset.

9 . The method of claim 1 , wherein the first software program and the second software program are analytics software usable for performing a statistical analysis of the input dataset.

10 . The method of claim 1 , wherein the graphical user interface is configured to allow the end user to create multiple testing programs for testing different functionality of the second software program by interacting with the output dataset displayed in the graphical user interface, wherein the graphical user interface is further configured to allow the end user to initiate execution of the testing programs to test the different functionality of the second software program, and wherein the test results include a plurality of results from executing the testing programs.

11 . The method of claim 1 , wherein the testing program is configured to perform a validation test to validate the output from the second software program against the particular truncated value-selected by the end user.

12 . A non-transitory computer-readable medium comprising program code that is executable by one or more processors for causing the one or more processors to perform operations including:

obtaining an output dataset generated by a first software program based on an input dataset, wherein the input dataset includes numerical values, wherein the output dataset includes analytical metrics generated based on the numerical values, and wherein the first software program is a first analytical program provided by an entity that is different from an end user;

generating a graphical user interface (GUI) that displays the output dataset to the end user;

detecting a selection by the end user of a particular value of the output dataset in the GUI for use as an expected output value during a subsequent test;

providing a first interface element of the GUI through which the end user can input a maximum number of significant figures for the particular value;

in response to receiving the maximum number of significant figures from the end user via the first interface element, generating a truncated value based on the particular value, wherein the truncated value is a version of the particular value as truncated to the maximum number of significant figures input by the end user;

providing a second interface element showing the truncated value, the second interface element being different from the first interface element;

providing a third interface element through which the end user can input a log relative error value, wherein the third interface element is different than the first and second interface elements;

receiving the log relative error value from the end user via the third interface element;

generating a testing program, for testing functionality of a second software program, by inserting the truncated value into testing logic of a predefined code template as the expected output value, wherein the second software program is a second analytical program, and wherein generating the testing program further involves inserting the log relative error value into the testing logic of the predefined code template;

executing the testing program to test the functionality of the second software program and thereby generate test results, wherein the testing program is configured to:

provide the input dataset as input to the second software program;

receive an output from the second software program generated based on the input dataset;

determine whether the output of the second software program deviates from the truncated value by more than a threshold amount; and

in response to determining that the output of the second software program deviates from the truncated value by more than the threshold amount, flag the deviation in the test results; and

outputting the test results to the end user in the GUI.

13 . The non-transitory computer-readable medium of claim 12 , wherein the first software program is the same as the second software program.

14 . The non-transitory computer-readable medium of claim 12 , wherein the first software program is a first version of a software program, and the second software program is a second version of the software program.

15 . The non-transitory computer-readable medium of claim 12 , wherein the operations further comprise:

based on the selection, automatically populating a fourth interface element of the graphical user interface with the particular value, the fourth interface element being editable by the end user, wherein the fourth interface element is different from the first, second, and third interface elements.

16 . The non-transitory computer-readable medium of claim 12 , wherein the graphical user interface includes a code preview window that shows source code of the testing program, the source code being editable by the end user in the code preview window to modify the testing program.

17 . The non-transitory computer-readable medium of claim 12 , wherein the operations further comprise:

selecting the predefined code template from among a plurality of predefined code templates based on a characteristic of the second software program, each code template of the plurality of predefined code templates being configured to test a different software program from the other code templates of the plurality of predefined code templates.

18 . The non-transitory computer-readable medium of claim 12 , wherein the operations further comprise:

detecting a plurality of selections by the end user of a plurality of locations in the graphical user interface corresponding to a plurality of values of the output dataset; and

based on the plurality of selections, configuring the testing program to include the plurality of values in the testing logic of the predefined code template.

19 . The non-transitory computer-readable medium of claim 12 , wherein the operations further comprise:

generating the output dataset by applying the first software program to the input dataset.

20 . The non-transitory computer-readable medium of claim 12 , wherein the first software program and the second software program are analytics software usable for performing a statistical analysis of the input dataset.

21 . The non-transitory computer-readable medium of claim 12 , wherein the graphical user interface is configured to allow the end user to create multiple testing programs for testing different functionality of the second software program by interacting with the output dataset displayed in the graphical user interface, wherein the graphical user interface is further configured to allow the end user to initiate execution of the testing programs to test the different functionality of the second software program, and wherein the test results include a plurality of results from executing the testing programs.

22 . The non-transitory computer-readable medium of claim 12 , wherein the testing program is configured to perform a validation test to validate the output from the second software program against the truncated value.

23 . A system comprising:

one or more processors; and

one or more memories comprising program code that is executable by the one or more processors for causing the one or more processors to perform operations including:

obtaining an output dataset generated by a first software program based on an input dataset, wherein the input dataset includes numerical values, wherein the output dataset includes analytical metrics generated based on the numerical values, and wherein the first software program is a first analytical program provided by an entity that is different from an end user;

generating a graphical user interface (GUI) that displays the output dataset to the end user;

detecting a selection by the end user of a particular value of the output dataset in the GUI for use as an expected output value during a subsequent test;

providing a first interface element of the GUI through which the end user can input a maximum number of significant figures for the particular value;

in response to receiving the maximum number of significant figures from the end user via the first interface element, generating a truncated value based on the particular value, wherein the truncated value is a version of the particular value as truncated to the maximum number of significant figures input by the end user;

providing a second interface element showing the truncated value, the second interface element being different from the first interface element;

providing a third interface element through which the end user can input a log relative error value, wherein the third interface element is different than the first and second interface elements;

receiving the log relative error value from the end user via the third interface element;

generating a testing program, for testing functionality of a second software program, by inserting the truncated value into testing logic of a predefined code template as the expected output value, wherein the second software program is a second analytical program, and wherein generating the testing program further involves inserting the log relative error value into the testing logic of the predefined code template;

executing the testing program to test the functionality of the second software program and thereby generate test results, wherein the testing program is configured to:

provide the input dataset as input to the second software program;

receive an output from the second software program generated based on the input dataset;

determine whether the output of the second software program deviates from the truncated value by more than a threshold amount; and

in response to determining that the output of the second software program deviates from the truncated value by more than the threshold amount, flag the deviation in the test results; and

outputting the test results to the end user in the GUI.

24 . The system of claim 23 , wherein the operations further comprise:

based on the selection, automatically populating a fourth interface element of the graphical user interface with the particular value, the fourth interface element being editable by the end user.

25 . The system of claim 23 , wherein the graphical user interface includes a code preview window that shows source code of the testing program, the source code being editable by the end user in the code preview window to modify the testing program.

26 . The system of claim 23 , wherein the operations further comprise:

selecting the predefined code template from among a plurality of predefined code templates based on a characteristic of the second software program, each code template of the plurality of predefined code templates being configured to test a different software program from the other code templates of the plurality of predefined code templates.

27 . The system of claim 23 , wherein the operations further comprise:

detecting a plurality of selections by the end user of a plurality of locations in the graphical user interface corresponding to a plurality of values of the output dataset; and

based on the plurality of selections, configuring the testing program to include the plurality of values in the testing logic of the predefined code template.

28 . The system of claim 23 , wherein the first software program and the second software program are analytics software usable for performing a statistical analysis of the input dataset.

29 . The system of claim 23 , wherein the graphical user interface is configured to allow the end user to create multiple testing programs for testing different functionality of the second software program by interacting with the output dataset displayed in the graphical user interface, wherein the graphical user interface is further configured to allow the end user to initiate execution of the testing programs to test the different functionality of the second software program, and wherein the test results include a plurality of results from executing the testing programs.

30 . The system of claim 23 , wherein the testing program is configured to perform a validation test to validate the output from the second software program against the truncated value.

Assignments (1)

Continuity (3)

References Cited (23)