IP Library › Granted Patent US 10,878,006

Granted Patent B2

US 10,878,006 · App. 15/883,734 · Granted Dec 29, 2020

Systems to interleave search results and related methods therefor

Inventor: Ergin Guney (San Francisco, CA)

Assignee: WALMART APOLLO LLC

G06F16/3325 G06F16/3334 G06F16/9535

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 10,878,006
App. No.: 15/883,734
Granted: Dec 29, 2020
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Some embodiments include a system. The system can comprise one or more processors, and one or more non-transitory memory storage devices storing computer instructions configured to run on the one or more processors and perform: receiving a base query string comprising a query parameter; and generating a modified version of the base query string to create a modified query string. The modified query string can comprise the query parameter and an interleaving parameter. Meanwhile, generating the modified version of the base query string to create the modified query string can comprise adding the interleaving parameter to the base query string. Other embodiments of related systems and methods are also provided.

Claims (204)

1. A system comprising:

one or more processors; and

one or more non-transitory memory storage devices storing computer instructions configured to run on the one or more processors and perform:

receiving a base query string comprising a query parameter;

generating a modified version of the base query string to create a modified query string;

determining first search results based on the query parameter; and

determining second search results based on the query parameter, wherein:

the modified query string comprises the query parameter and an interleaving parameter;

generating the modified version of the base query string to create the modified query string comprises:

adding the interleaving parameter to the base query string; and

the first search results and the second search results are:

determined by a same search algorithm; and

at least partially different from each other.

2. The system of claim 1 wherein:

the computer instructions further are configured to perform:

determining that the base query string is associated with a control user; and

when the base query string is determined to be associated with the control user, generating the modified version of the base query string to create the modified query string further comprises:

setting the interleaving parameter to be inactive.

3. The system of claim 1 wherein:

the computer instructions further are configured to perform:

determining that the base query string is associated with a test user; and

when the base query string is determined to be associated with the test user, generating the modified version of the base query string to create the modified query string further comprises:

setting the interleaving parameter to be active.

4. The system of claim 1 wherein:

the computer instructions further are configured to perform:

reviewing the modified query string, wherein reviewing the modified query string comprises:

determining that the interleaving parameter is active; and

responsive to determining that the interleaving parameter is active:

generating a first sub-query string based on the query parameter; and

generating a second sub-query string comprising based on the query parameter and a test parameter;

determining the first search results based on the query parameter comprises:

determining the first search results based on the first sub-query string, which is based on the query parameter; and

determining the second search results based on the query parameter comprises:

determining the second search results based on the second sub-query string, which is based on the query parameter and the test parameter; and

generating interleaved search results, wherein generating the interleaved search results comprises:

interleaving together the first search results and the second search results for an output for a search query associated with the base query string.

5. The system of claim 4 wherein:

interleaving together the first search results and the second search results for the output for the search query associated with the base query string comprises:

team-draft interleaving together the first search results and the second search results for the output for the search query associated with the base query string.

6. The system of claim 4 wherein:

interleaving together the first search results and the second search results for the output for the search query associated with the base query string comprises:

probabilistic interleaving together the first search results and the second search results for the output for the search query associated with the base query string.

7. The system of claim 1 wherein:

the modified query string further comprises an experiment parameter;

the experiment parameter is associated with a test parameter;

generating the modified version of the base query string to create the modified query string further comprises:

adding the experiment parameter to the base query string;

the computer instructions further are configured to perform:

reviewing the modified query string, wherein reviewing the modified query string comprises:

determining that the interleaving parameter is active; and

determining that the experiment parameter is active; and

responsive to determining that the interleaving parameter is active, and responsive to determining that the experiment parameter is active:

generating a first sub-query string based on the query parameter; and

generating a second sub-query string based on the query parameter and the test parameter;

determining the first search results based on the query parameter comprises:

determining the first search results based on the first sub-query string, which is based on the query parameter; and

determining the second search results based on the query parameter comprises:

determining the second search results of the second sub-query string, which is based on the query parameter and the test parameter; and

generating interleaved search results, wherein generating the interleaved search results comprises:

interleaving together the first search results and the second search results for an output for a search query associated with the base query string.

8. The system of claim 1 wherein:

the modified query string further comprises a first experiment parameter and a second experiment parameter;

the first experiment parameter is associated with a first test parameter;

the second experiment parameter is associated with a second test parameter different than the first test parameter;

generating the modified version of the base query string to create the modified query string further comprises:

adding the first experiment parameter to the base query string; and

adding the second experiment parameter to the base query string;

the computer instructions further are configured to perform:

reviewing the modified query string, wherein reviewing the modified query string comprises:

determining that the interleaving parameter is active; and

at least one of:

determining that the first experiment parameter is active; or

determining that the second experiment parameter is active;

when reviewing the modified query string comprises determining that the first experiment parameter is active and does not comprise determining that the second experiment parameter is active, responsive to determining that the interleaving parameter is active, and responsive to determining that the first experiment parameter is active:

generating a first sub-query string based on the query parameter;

generating a second sub-query string based on the query parameter and the first test parameter, wherein:

determining the first search results based on the query parameter comprises:

determining the first search results based on the first sub-query string, which is based on the query parameter; and

determining the second search results based on the query parameter comprises:

determining the second search results based on the second sub-query string, which is based on the query parameter and the first test parameter; and

generating first interleaved search results, wherein generating the first interleaved search results comprises:

interleaving together the first search results and the second search results for an output for a search query associated with the base query string; and

when reviewing the modified query string does not comprise determining that the first experiment parameter is active and does comprise determining that the second experiment parameter is active, responsive to determining that the interleaving parameter is active, and responsive to determining that the second experiment parameter is active:

generating the first sub-query string based on the query parameter;

generating a third sub-query string based on the query parameter and the second test parameter, wherein:

determining the first search results based on the query parameter comprises:

determining the first search results based on the first sub-query string, which is based on the query parameter; and

determining the second search results based on the query parameter comprises:

determining third search results based on the third sub-query string, which is based on the query parameter and the second test parameter; and

generating second interleaved search results, wherein generating the second interleaved search results comprises:

interleaving together the first search results and the third search results for the output for the search query associated with the base query string.

9. The system of claim 1 , wherein the computer instructions are further configured to perform:

disabling a cache of the system prior to determining the first search results and the second search results.

10. A method, the method being implemented via execution of computer instructions configured to run at one or more processors and configured to be stored at one or more non-transitory memory storage devices, and comprising:

receiving a base query string comprising a query parameter;

generating a modified version of the base query string to create a modified query string;

determining first search results based on the query parameter; and

determining second search results based on the query parameter, wherein:

the modified query string comprises the query parameter and an interleaving parameter;

generating the modified version of the base query string to create the modified query string comprises:

adding the interleaving parameter to the base query string; and

the first search results and the second search results are:

determined by a same search algorithm; and

at least partially different from each other.

11. The method of claim 10 further comprising:

determining that the base query string is associated with a control user;

wherein:

when the base query string is determined to be associated with the control user, generating the modified version of the base query string to create the modified query string further comprises:

setting the interleaving parameter to be inactive.

12. The method of claim 10 further comprising:

determining that the base query string is associated with a test user;

wherein:

when the base query string is determined to be associated with the test user, generating the modified version of the base query string to create the modified query string further comprises:

setting the interleaving parameter to be inactive.

13. The method of claim 10 , wherein:

the method further comprises:

reviewing the modified query string, wherein reviewing the modified query string comprises:

determining that the interleaving parameter is active; and

responsive to determining that the interleaving parameter is active:

generating a first sub-query string based on the query parameter; and

generating a second sub-query string based on the query parameter and a test parameter;

determining the first search results based on the query parameter comprises:

determining the first search results based on the first sub-query string, which is based on the query parameter;

determining the second search results based on the query parameter comprises:

determining the second search results based on the second sub-query string, which is based on the query parameter and the test parameter; and

generating interleaved search results, wherein generating the interleaved search results comprises:

interleaving together the first search results and the second search results for an output for a search query associated with the base query string.

14. The method of claim 13 wherein:

interleaving together the first search results and the second search results for the output for the search query associated with the base query string comprises:

team-draft interleaving together the first search results and the second search results for the output for the search query associated with the base query string.

15. The method of claim 13 wherein:

interleaving together the first search results and the second search results for the output for the search query associated with the base query string comprises:

probabilistic interleaving together the first search results and the second search results for the output for the search query associated with the base query string.

16. The method of claim 10 wherein:

the modified query string further comprises an experiment parameter;

the experiment parameter is associated with a test parameter;

generating the modified version of the base query string to create the modified query string further comprises:

adding the experiment parameter to the base query string;

the method further comprises:

reviewing the modified query string, wherein reviewing the modified query string comprises:

determining that the interleaving parameter is active; and

determining that the experiment parameter is active;

and