Organizing search results
View Patent ↗At least one listing is retrieved based on a search query, the query including at least one parameter associated with an element of a hierarchical index A determination of a normalized click-through rate for each of the at least one listings is made, the normalized click-through rate for a listing being based on a click-through rate for the listing that is adjusted according to one or more positions in which the listing has been displayed in each of a plurality of sets of search results. A determination is made of a position for the at least one listing in a set of search results based at least in part on an aggregation of the aggregate normalized click-through rates associated with at least one level of a hierarchical index.
1. A Non-Transitory computer-readable medium tangibly storing instructions executable on one or more computers, the instructions including instructions for:
retrieving at least one listing based on a search query, the query including at least one parameter associated with an element of a hierarchical index;
determining a normalized click-through rate for each of the at least one listings,
the normalized click-through rate for a listing being based on a click-through rate for the listing that is adjusted according to one or more positions in which the listing has been displayed in each of a plurality of sets of search results; and
determining a position for the at least one listing in a set of search results based at least in part on an aggregation of the normalized click-through rates associated with at least one level of a hierarchical index;
aggregating normalized click-through rates associated with a first level of a hierarchical index into an aggregate normalized click-through rate for the first level;
aggregating normalized click-through rates associated with a second level of a hierarchical index into an aggregate normalized click-through rate for the second level;
determining a deviation of the difference between the second-level normalized click-through rates and the first-level normalized click-through rates;
comparing the deviation with a predetermined threshold; and
accepting the first-level normalized click-through rate as a valid aggregation when the deviation is below the predetermined threshold.
2. The Non-Transitory computer-readable medium of claim 1 , wherein the hierarchical index is a plurality of hierarchical indices.
3. A method, comprising:
identifying a listing; and
determining, in a computer, a click-through rate for the listing based at least in part on:
at least one element within a first hierarchical index with which the listing is associated; and
a count of the number of times the listing has been selected by a user when displayed at a particular position at which the listing was displayed in a set of search results.
4. The method of claim 3 , further comprising adjusting the click-through rate for the listing based on positions in which the listing was displayed in sets of search results.
5. The method of claim 4 , further comprising adjusting the click-through rate for the listing to indicate a click-through rate for a listing displayed at a given position.
6. The method of claim 5 , wherein the given position is one of a first position, second position, third position, fourth position, or fifth position.
7. The method of claim 4 , further comprising adjusting the click-through rate for the listing according to the formula:
Norm_CTR
hierarchy
=
(
∑
i
=
0
n
Avg_CTR
0
*
CTR
i
,
hierarchy
/
Avg_CTR
i
)
/
n
,
where:
Avg_CTR 0 is the average click-through rate for a control set of pay-per-click listings that have been provided in the first position in the sets of search results;
Avg_CTR i is the average click-through rate for a control set of pay-per-click listings that have been provided in the ith position in the sets of search results;
CTR i,hierarchy is the click-through rate of the listing of interest associated with a given element within the first hierarchy and at the ith position; and
n is the total number of possible positions considered.
8. The method of claim 3 , further comprising determining the click-through rate based at least in part on at least one element within a second hierarchical index with which the listing is associated.
9. The method of claim 8 , wherein the first hierarchical index is a hierarchical geography index, and the second hierarchical index is a hierarchical category index.
10. The method of claim 9 , further comprising adjusting the click-through rate for the listing according to the formula:
(
∑
i
=
0
n
Avg_CTR
(
0
,
geography
,
category
)
*
CTR
(
i
,
geography
,
category
,
campaign
)
Avg_CTR
(
i
,
geography
,
category
)
)
/
n
,
where:
CTR(i, geography, category, campaign) is the click-through rate of a campaign associated with a given category within the hierarchical category index and associated with a given geography within the hierarchical geography index and at position i;
Avg_CTR(0, geography, category) is the average click-through rate of all campaigns at position 0 associated with the given category and the given geography; and
Avg_CTR(i, geography, category) is the average click-through rate of all campaigns at the position i associated with the given category and the given geography, and n is the possible number of distinct positions occupied by the listing.
11. A method, comprising:
retrieving or calculating normalized click-through rates for a first level of a hierarchical index;
aggregating, in a computer, the normalized click-through rates that fall under a second level of the hierarchical index into an aggregate normalized click-through rate for the second level;
determining a statistical deviation of the difference between the second-level normalized click-through rates and the first-level normalized click-through rates;
comparing the deviation with a predetermined threshold; and
accepting the first-level normalized click-through rate as a valid aggregation when the deviation is below the predetermined threshold.
12. The method of claim 11 , further comprising sorting a plurality of valid aggregate click-through rates to determine the highest level of acceptable aggregation.
13. The method of claim 12 , wherein the hierarchical index is a geographical hierarchy.
14. The method of claim 13 , wherein the geographical levels of the geographical hierarchy are organized according to at least one of proximity, demography, and size.
15. The method of claim 12 , further comprising ordering a set of listings based on the click-through rates at the highest level of acceptable aggregation.
16. The method of claim 15 , wherein the set of listings includes at least one pay-per-click listing.
17. The method of claim 11 , wherein the deviation is a standard deviation.
18. The method of claim 11 , further comprising adjusting the click-through rates based on one or more positions in which listings have been displayed in each of a plurality of sets of search results.
19. The method of claim 18 , further comprising basing the normalized click-through rates on actual click-through rates adjusted to indicate click-through rates for listings at a given position.
20. The method of claim 19 , wherein normalizing the click-through rates further comprises using the formula:
(
∑
i
=
0
n
Avg_CTR
(
0
,
geography
,
category
)
*
CTR
(
i
,
geography
,
category
,
campaign
)
Avg_CTR
(
i
,
geography
,
category
)
)
/
n
,
where:
CTR(i, geography, category, campaign) is the click-through rate of a campaign with a given category within a hierarchical category index and a given geography within a hierarchical geography index and at position i;
Avg_CTR(0, geography, category) is the average click-through rate of all campaigns at position 0 with the given category and the given geography; and
Avg_CTR(i, geography, category) is the average click-through rate of all campaigns at the position i with the given category and the given geography and n is the possible number of distinct positions occupied by the listing.