Granted Patent B1

US 12,726,449 · App. 19/232,798 · Granted Sep 1, 2026

Inventors: Matthew Isaac van Adelsberg (Falls Church, VA); Zhun Wang (Vienna, VA); Priscilla Alexander (Washington, DC); Mackenzie Sweeney (Gainesville, VA)

Assignee: Capital One Services, LLC

H04L47/821H04L47/24

Abstract

Claims (76)

1 . A system, comprising:

memory; and

one or more processors, coupled to memory, configured to cause the system to:

receive, from a client device, a request to access functionality of a digital testing agent to test a website associated with the client device;

grant the client device the access to the functionality of the digital testing agent before the client device uploads variants, associated with the website, to a web server for digital testing;

receive, from the web server and via one or more application programming interface (API) endpoints, data that indicates:

a variant, of the variants, served to a user device, and

one or more of:

whether the user device converted,

a device type associated with the user device, or interactions with the variant via the user device;

identify, based on the data, that the variant is overperforming in relation to one or more other variants of the variants;

adjust, based on the variant overperforming in relation to the one or more other variants, the digital testing by continuously allocating volume to higher performing variants of the variants, wherein the higher performing variants include the variant;

adjust, based on a response rate change associated with the variants, the digital testing by injecting extra explorations into the digital testing;

receive, from the client device, a request to halt testing of a webpage of the website; and

halt testing of the website without disrupting one or more results or recommendations from the testing of the website.

2 . The system of claim 1 ,

wherein the data indicates that the user device converted, and

wherein, to identify that the variant is overperforming in relation to the one or more other variants, the one or more processors cause the system to:

identify, based on the variant being served to the user device and based on data indicating that the user device converted, that the variant is overperforming in relation to the one or more other variants.

3 . The system of claim 1 ,

wherein the data indicates the device type associated with the user device, and

wherein, to identify that the variant is overperforming in relation to the one or more other variants, the one or more processors cause the system to:

identify, based on the variant being served to the user device and based on the device type associated with the user device, that the variant is overperforming in relation to the one or more other variants.

4 . The system of claim 1 ,

wherein the data indicates the interactions with the variant via the user device, and

wherein, to identify that the variant is overperforming in relation to the one or more other variants, the one or more processors cause the system to:

identify, based on the variant being served to the user device and based on the interactions with the variant via the user device, that the variant is overperforming in relation to the one or more other variants.

5 . The system of claim 1 , wherein the one or more processors further cause the system to:

cause a pause to the digital testing before identifying that the variant is overperforming in relation to the one or more other variants.

6 . The system of claim 1 , wherein, to identify that the variant is overperforming in relation to the one or more other variants, the one or more processors cause the system to:

input the data into one or more machine learning models to generate one or more results of testing the website, and

identify, based on the one or more results, that the variant is overperforming in relation to the one or more other variants.

7 . The system of claim 6 , wherein the one or more results quantify one or more of risks or benefits associated with the variant.

8 . The system of claim 1 , wherein, to adjust the digital testing of the variants, the one or more processors further cause the system to:

determine to cease operation of a different variant of the one or more other variants based on the variant overperforming in relation to the one or more other variants.

9 . A method, comprising:

receiving a request to test a website associated with a client device;

granting the request before the client device uploads variants, associated the website, to a web server;

receiving, from the web server and via one or more application programming interface (API) endpoints, data that indicates:

a variant, of the variants, served to a user device, and

one or more of whether the user device converted, a device type associated with the user device, or interactions with the variant via the user device;

determining a response rate change in relation to one or more other variants of the variants;

adjusting, based on the response rate change in relation to the one or more other variants, testing of the variants by injecting extra explorations into the testing;

receiving a request to halt the testing; and

halting the testing of the website without disrupting one or more results or recommendations from the testing.

10 . The method of claim 9 , further comprising:

determining to cease operation of a different variant of the variants based on the variant overperforming in relation to the different variant; and

allocating network traffic to the variant based on ceasing the operation of the different variant.

11 . The method of claim 9 , further comprising:

directing network traffic to one or more endpoints associated with a different variant, of the one or more other variants, based on the different variant performing better than the variant.

12 . The method of claim 9 , further comprising:

determining that the variant is overperforming in relation to the one or more other variants.

13 . The method of claim 12 , further comprising:

allocating network traffic to the variant based on the variant overperforming in relation to the one or more other variants.

14 . The method of claim 13 , further comprising:

determining, after allocating the network traffic to the variant, that a different variant, of the variants, is performing better than the variant.

15 . The method of claim 14 , further comprising:

adjusting the testing of the variants another time after determining that the different variant is performing better than the variant.

16 . The method of claim 15 , wherein adjusting the testing of the variants another time comprises:

allocating new network traffic to the different variant based on determining that the different variant is performing better than the variant.

17 . The method of claim 9 ,

wherein the data indicates the interactions with the variant via the user device, and

wherein the method further comprises:

identifying, based on the variant being served to the user device and based on the interactions with the variant via the user device, that the variant is overperforming in relation to the one or more other variants.

18 . One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause one or more operations comprising:

receiving, from a web server and via one or more application programming interface (API) endpoints, data that indicates:

a variant served to a user device, and

one or more of whether the user device converted, a device type associated with the user device, or interactions with the variant via the user device;

determining, based on the data, that the variant is performing better than one or more other variants;

adjusting, based on determining that the variant is performing better in relation to the one or more other variants, testing by continuously allocating volume to one or more higher performing variants that include the variant;

receiving a request to halt the testing; and

halting the testing without disrupting one or more results or recommendations from the testing.

19 . The one or more non-transitory computer readable media of claim 18 , wherein the one or more operations further comprise:

adjusting the testing another time by allocating network traffic to a different variant based on determining that the different variant is performing better than the variant.

20 . The one or more non-transitory computer readable media of claim 18 , wherein the one or more operations further comprise:

granting functionality of a digital testing agent to upload the variant for the testing.

Assignments (1)

Continuity (2)

References Cited (10)