IP Library Granted Patent US 11,023,958
Granted Patent B2
US 11,023,958 · App. 15/883,008 · Granted Jun 1, 2021

Smart measurement points

Inventors: Anthony Tang (San Diego, CA); Swetharam Kasireddi (Santa Clara, CA); Justin Rubin (San Diego, CA); Gholamreza Esfandani Bozchelouei (Carlsbad, CA); Girish Subramanian (San Diego, CA); Duy Le (San Diego, CA); Shriram Sharma (San Diego, CA); Stephen Bitondo (Carlsbad, CA)
Assignee: Walmart Apollo, LLC
G06Q30/0641G06F9/451G06F16/9038G06Q30/0201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,023,958
App. No.
15/883,008
Granted
Jun 1, 2021
Kind
B2
Abstract

Systems and methods for dynamically defining smart measurement points are disclosed. A first experiment configured to provide one of a base element or a variant element is defined and the first experiment is associated with a first page. The first experiment includes a first dynamically defined qualifying measurement point. A plurality of records each comprising one or more variables indicative of an interaction between one of the plurality of user systems and the computing device are generated and a search query including the first dynamically defined qualifying measurement point is received. Each record received is searched. For each record containing the first dynamically defined qualifying measurement point, an associated one of the plurality of user devices is qualified for the at least one experiment and, for each remaining record, the associated one of the plurality of user devices is excluded for the at least one experiment.

Claims (53)

1. A system, comprising:

a computing device communicatively coupled to a database, where the computing device is configured to:

receive a first experiment configured to provide each user system of a plurality of user systems one of a base element or a variant element, wherein the first experiment is associated with a first page, and wherein the first experiment includes a first dynamically defined qualifying measurement point indicating a qualifying event corresponding to one or more user interactions of a corresponding user system of the plurality of user systems with the first page;

generate a plurality of records based on user interaction data stored within the database, each record comprising one or more variables indicative of an interaction between one of the plurality of user systems and the computing device;

receive a search query including the first dynamically defined qualifying measurement point;

search each record of the plurality of records received at the computing device, wherein:

for each record containing the first dynamically defined qualifying measurement point, qualify the corresponding one of the plurality of user systems for the first experiment; and

for each remaining record, exclude the corresponding one of the plurality of user systems for the first experiment;

analyze the variant element based on records corresponding to qualified user systems of the plurality of user systems; and

generate one or more outputs based on the analysis of the variant element.

2. The system of claim 1 , wherein the first experiment includes at least one reportable measurement point, and wherein the computing device is further configured to compare the qualified user systems to the at least one reportable measurement points.

3. The system of claim 1 , wherein the first experiment includes at least one system defined measurement point, wherein each record containing the first dynamically defined qualifying measurement point is qualified for the first experiment only when the record includes the at least one system defined measurement point.

4. The system of claim 1 , wherein the computing device is further configured to receive a second dynamically defined measurement point.

5. The system of claim 4 , wherein the second dynamically defined measurement point is defined after at least one user system has qualified for the first experiment.

6. The system of claim 4 , wherein, for each record containing the second dynamically defined measurement point, the computing device is configured to exclude the corresponding one of the plurality of user systems from the first experiment.

7. The system of claim 1 , wherein the computing device is further configured to:

review a syntax of the first dynamically defined measurement point; and

generate an alert when the syntax of the first dynamically defined measurement point fails to qualify any user system.

8. The system of claim 1 , wherein the computing device is further configured to:

define a second experiment configured to provide one of a second base element or a second variant element, wherein the second experiment is associated with a second page, and wherein the second experiment includes a second dynamically defined qualifying measurement point;

receive a search query including the second dynamically defined qualifying measurement point;

search each record received at the computer system, wherein:

for each record containing the second dynamically defined qualifying measurement point, qualify corresponding one of the plurality of user systems for the second experiment, wherein the second dynamically defined qualifying measurement point is different than the first dynamically defined qualifying measurement point.

9. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by a processor cause a device to perform operations comprising:

receiving a first experiment configured to provide each user system of a plurality of user systems one of a base element or a variant element, wherein the first experiment is associated with a first page, and wherein the first experiment includes a first dynamically defined qualifying measurement point indicating a qualifying event corresponding to one or more user interactions of a corresponding user system of the plurality of user systems with the first page;

generating a plurality of records based on user interaction data, each record comprising one or more variables indicative of an interaction between one of a plurality of user systems and the computing device;

receiving a search query including the first dynamically defined qualifying measurement point;

searching each record of the plurality of records, wherein:

for each record containing the first dynamically defined qualifying measurement point, qualify corresponding one of the plurality of user systems for the first experiment; and

for each remaining record, exclude the corresponding one of the plurality of user systems for the first experiment.

10. The non-transitory computer readable medium of claim 9 , wherein the first experiment includes at least one reportable measurement point, and wherein the method further comprises comparing the qualified user systems to the at least one reportable measurement points.

11. The non-transitory computer readable medium of claim 9 , wherein the first experiment includes at least one system defined measurement point, wherein each record containing the first dynamically defined qualifying measurement point is qualified for the first experiment only when the record includes the at least one system defined measurement point.

12. The non-transitory computer readable medium of claim 9 , comprising receiving a second dynamically defined measurement point.

13. The non-transitory computer readable medium of claim 12 , wherein the second dynamically defined measurement point is defined after at least one user system has qualified for the first experiment.

14. The non-transitory computer readable medium of claim 12 , wherein, for each record containing the second dynamically defined measurement point, the corresponding one of the plurality of user systems is excluded from the first experiment.

15. The non-transitory computer readable medium of claim 9 , wherein the computing device is further configured to:

review a syntax of the first dynamically defined measurement point; and

generate an alert when the syntax of the first dynamically defined measurement point fails to qualify any user system.

16. The non-transitory computer readable medium of claim 9 , wherein the instructions further cause the device to perform operations comprising:

defining a second experiment configured to provide one of a second base element or a second variant element, wherein the second experiment is associated with a second page, and wherein the second experiment includes a second dynamically defined qualifying measurement point;

receiving a search query including the second dynamically defined qualifying measurement point;

searching each record, wherein:

for each record containing the second dynamically defined qualifying measurement point, qualify corresponding one of the plurality of user systems for the first experiment, wherein the second dynamically defined qualifying measurement point is different than the first dynamically defined qualifying measurement point.

17. A method, comprising:

receiving a first experiment configured to provide each user system of a plurality of user systems one of a base element or a variant element, wherein the first experiment is associated with a first page, and wherein the first experiment includes a first dynamically defined qualifying measurement point indicating a qualifying event corresponding to one or more user interactions of a corresponding user system of the plurality of user systems with the first page;

generating a plurality of records based on user interaction data, each record comprising one or more variables indicative of an interaction between one of a plurality of user systems and the computing device;

receiving a search query including the first dynamically defined qualifying measurement point;

searching each record of the plurality of records received, wherein:

for each record containing the first dynamically defined qualifying measurement point, qualify corresponding one of the plurality of user systems for the first experiment; and

for each remaining record, exclude the corresponding one of the plurality of user systems for the first experiment.

18. The method of claim 17 , wherein the first experiment includes at least one reportable measurement point, and wherein the method further comprises comparing the qualified user systems at the at least one reportable measurement points.

19. The method of claim 17 , wherein the first experiment includes at least one system defined measurement point, wherein each record containing the first dynamically defined qualifying measurement point is qualified for the first experiment only when the record includes the at least one system defined measurement point.

20. The method of claim 17 , comprising receiving a second dynamically defined measurement point.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 29, 2018
From: WAL-MART STORES, INC.
To: WALMART APOLLO, LLC
Reel/Frame 045392/0554 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 29, 2018
From: TANG, ANTHONY; KASIREDDI, SWETHARAM; RUBIN, JUSTIN; BOZCHELOUEI, GHOLAMREZA ESFANDANI; SUBRAMANIAN, GIRISH; LE, DUY; SHARMA, SHRIRAM; BITONDO, STEPHEN
To: WAL-MART STORES, INC.
Reel/Frame 044760/0409 →
Continuity (1)
Related Publication 20190236687A1 · Aug 1, 2019