Click or skip evaluation of query term optionalization rule
View Patent ↗Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for click or skip evaluation of optionalization rules. In one aspect, a method includes accessing query log data that identifies, for an initial search query, one or more revised search queries that were identified by designating a query term of the initial search query as optional according to one or more optionalization rules, and a particular search result selected by the user. A determination is made, using the query log data, that the particular search result corresponds to the one or more revised search queries that were identified by designating the query term of the initial search query as optional according to one or more optionalization rules. In response to this determination, a click count is incremented for the one or more optionalization rules that corresponds to the particular search result.
1. A computer-implemented method comprising:
accessing data that identifies one or more query term optionalization rules that are specific to a particular query term, the one or more query term optionalization rules indicating whether the particular query term should be made optional in revisions of search queries that include the particular query term;
accessing query log data that identifies, for an initial search query that includes the particular query term:
one or more revised search queries that a search engine system generated in response to the initial search query using the one or more query term optionalization rules that are specific to the particular query term, wherein the particular query term is optional in the one or more revised search queries,
search results that were identified using the initial search query,
search results that were identified using the one or more revised search queries, and
a particular search result selected by a user from among a set of search results that includes (i) one or more of the search results that were identified using the initial search query and (ii) one or more of the search results that were identified using the one or more revised search queries;
determining, using the query log data, that the particular search result is one of the search results that were identified using the one or more revised search queries;
in response to determining that the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term; and
after adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term, determining, based at least on the click count for the one or more query term optionalization rules that are specific to the particular query term, whether to revise a subsequently-received search query that includes the particular query term using the one or more query term optionalization rules that are specific to the particular query term.
2. The method of claim 1 , comprising:
ranking the search results included in the set of search results that includes (i) the one or more of the search results that were identified using the initial search query and (ii) the one or more of the search results that were identified using the one or more revised search queries;
determining, using the query log data, that a search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries; and
in response to determining that the search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a skip count for the one or more query term optionalization rules that are specific to the particular query term.
3. The method of claim 2 , comprising associating a first weight with the click count and a second weight with the skip count.
4. The method of claim 3 , wherein the score assigned to the one or more query term optionalization rules that are specific to the particular query term satisfies:
W
1
(
click
count
)
W
1
(
click
count
)
+
W
2
(
skip
count
)
wherein W1 represents the weight associated with the click count and W2 represents the weight associated with the skip count.
5. The method of claim 2 , comprising assigning a score to the one or more query term optionalization rules that are specific to the particular query term based at least on the click count and the skip count.
6. The method of claim 5 , wherein the score assigned to the one or more query term optionalization rules that are specific to the particular query term is based on a ratio of (i) the click count to (ii) the click count and the skip count.
7. The method of claim 5 , comprising:
determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy a threshold; and
removing the one or more query term optionalization rules that are specific to the particular query term from a set of query term optionalization rules that includes other query term optionalization rules that are specific to other query terms, the other query term optionalization rules indicating whether the other query terms should be made optional in revisions of search queries that include the other query terms, based on determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy the threshold.
8. The method of claim 1 , wherein the one or more revised search queries were generated by the search engine system using one or more query term optionalization rules that are specific to the particular query term and are associated with a particular context,
wherein the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context indicate whether the particular query term should be made optional in revisions of search queries that include the particular query term and the particular other query term,
wherein the initial search query includes the particular other query term, and
wherein adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term comprises adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context.
9. The method of claim 1 , wherein adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term comprises:
determining, using the query log data, that the particular search result does not include the particular query term; and
adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term in response to (i) determining that the particular search result is one of the search results that were identified using the one or more revised search queries, and (ii) determining that the particular search result does not include the particular query term included in the initial search query.
10. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
accessing data that identifies one or more query term optionalization rules that are specific to a particular query term, the one or more query term optionalization rules indicating whether the particular query term should be made optional in revisions of search queries that include the particular query term;
accessing query log data that identifies, for an initial search query that includes the particular query term:
one or more revised search queries that a search engine system generated in response to the initial search query using the one or more query term optionalization rules that are specific to the particular query term, wherein the particular query term is optional in the one or more revised search queries,
search results that were identified using the initial search query,
search results that were identified using the one or more revised search queries, and
a particular search result selected by a user from among a set of search results that includes (i) one or more of the search results that were identified using the initial search query and (ii) one or more of the search results that were identified using the one or more revised search queries;
determining, using the query log data, that the particular search result is one of the search results that were identified using the one or more revised search queries;
in response to determining that the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term; and
after adjusting the click count for the one or more query term optionalization rules that are specific to the particular query team, determining, based at least on the click count for the one or more query term optionalization rules that are specific to the particular query term, whether to revise a subsequently-received search query that includes the particular query term using the one or more query term optionalization rules that are specific to the particular query term.
11. The medium of claim 10 , wherein the operations comprise:
ranking the search results included in the set of search results that includes (i) the one or more of the search results that were identified using the initial search query and (ii) the one or more of the search results that were identified using the one or more revised search queries;
determining, using the query log data, that a search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries; and
in response to determining that the search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a skip count for the one or more query term optionalization rules that are specific to the particular query term.
12. The medium of claim 11 , wherein the operations comprise assigning a score to the one or more query term optionalization rules that are specific to the particular query term based at least on the click count and the skip count.
13. The medium of claim 12 , wherein the score assigned to the one or more query term optionalization rules that are specific to the particular query term is based on a ratio of (i) the click count to (ii) the click count and the skip count.
14. The medium of claim 12 , wherein the operations comprise:
determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy a threshold; and
removing the one or more query term optionalization rules that are specific to the particular query term from a set of query term optionalization rules that includes other query term optionalization rules that are specific to other query terms, the other query term optionalization rules indicating whether the other query terms should be made optional in revisions of search queries that include the other query terms, based on determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy the threshold.
15. The medium of claim 11 , wherein the operations comprise associating a first weight with the click count and a second weight with the skip count.
16. The medium of claim 15 , wherein the score assigned to the one or more query term optionalization rules that are specific to the particular query term satisfies:
W
1
(
click
count
)
W
1
(
click
count
)
+
W
2
(
skip
count
)
wherein W1 represents the weight associated with the click count and W2 represents the weight associated with the skip count.
17. The medium of claim 10 , wherein the one or more revised search queries were generated by the search engine system using one or more query term optionalization rules that are specific to the particular query term and are associated with a particular context,
wherein the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context indicate whether the particular query term should be made optional in revisions of search queries that include the particular query term and the particular other query term,
wherein the initial search query includes the particular other query term, and
wherein adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term comprises adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context.
18. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
accessing data that identifies one or more query term optionalization rules that are specific to a particular query term, the one or more query term optionalization rules indicating whether the particular query term should be made optional in revisions of search queries that include the particular query term;
accessing query log data that identifies, for an initial search query that includes the particular query term:
one or more revised search queries that a search engine system generated in response to the initial search query using the one or more query term optionalization rules that are specific to the particular query term, wherein the particular query term is optional in the one or more revised search queries,
search results that were identified using the initial search query,
search results that were identified using the one or more revised search queries, and
a particular search result selected by a user from among a set of search results that includes (i) one or more of the search results that were identified using the initial search query and (ii) one or more of the search results that were identified using the one or more revised search queries;
determining, using the query log data, that the particular search result is one of the search results that were identified using the one or more revised search queries;
in response to determining that the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term; and
after adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term, determining, based at least on the click count for the one or more query term optionalization rules that are specific to the particular query term, whether to revise a subsequently-received search query that includes the particular query term using the one or more query term optionalization rules that are specific to the particular query term.
19. The system of claim 18 , wherein the operations comprise:
ranking the search results included in the set of search results that includes (i) the one or more of the search results that were identified using the initial search query and (ii) the one or more of the search results that were identified using the one or more revised search queries;
determining, using the query log data, that a search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries; and
in response to determining that the search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a skip count for the one or more query term optionalization rules that are specific to the particular query term.
20. The system of claim 19 , wherein the operations comprise assigning a score to the one or more query term optionalization rules that are specific to the particular query term based at least on the click count and the skip count.
21. The system of claim 20 , wherein the operations comprise:
determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy a threshold; and
removing the one or more query term optionalization rules that are specific to the particular query term from a set of query term optionalization rules that includes other query term optionalization rules that are specific to other query terms, the other query term optionalization rules indicating whether the other query terms should be made optional in revisions of search queries that include the other query terms, based on determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy the threshold.
22. The system of claim 19 , wherein the operations comprise associating a first weight with the click count and a second weight with the skip count.
23. The system of claim 22 , wherein the score assigned to the one or more query term optionalization rules that are specific to the particular query term satisfies:
W
1
(
click
count
)
W
1
(
click
count
)
+
W
2
(
skip
count
)
wherein W1 represents the weight associated with the click count and W2 represents the weight associated with the skip count.
24. The system of claim 18 , wherein the one or more revised search queries were generated by the search engine system using one or more query term optionalization rules that are specific to the particular query term and are associated with a particular context,
wherein the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context indicate whether the particular query term should be made optional in revisions of search queries that include the particular query term and the particular other query term,
wherein the initial search query includes the particular other query term, and
wherein adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term comprises adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context.
25. A computer-implemented method comprising:
accessing data that identifies one or more query term optionalization rules that are specific to a particular query term, the one or more query term optionalization rules indicating whether the particular query term should be made optional in revisions of search queries that include the particular query term;
accessing query log data that identifies, for an initial search query that includes the particular query term:
one or more revised search queries that a search engine system generated in response to the initial search query using the one or more query term optionalization rules that are specific to the particular query term, wherein the particular query term is optional in the one or more revised search queries
search results that were identified using the initial search query,
search results that were identified using the one or more revised search queries,
a ranking of a set of search results that includes (i) one or more of the search results that were identified using the initial search query and (ii) one or more of the search results that were identified using the one or more revised search queries, and
a particular search result selected by a user from among the set of search results that includes (i) the one or more of the search results that were identified using the initial search query and (ii) the one or more of the search results that were identified using the one or more revised search queries;
determining, using the query log data, that a search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries;
in response to determining that the search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a skip count for the one or more query term optionalization rules that are specific to the particular query term; and
after adjusting the skip count for the one or more query term optionalization rules that are specific to the particular query term, determining, based at least on the skip count for the one or more query term optionalization rules that are specific to the particular query term, whether to revise a subsequently-received search query that includes the particular query term using the one or more query term optionalization rules that are specific to the particular query term.
26. The method of claim 25 , comprising:
determining, using the query log data, that the particular search result is one of the search results that were identified using the one or more revised search queries; and
in response to determining that the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term.
27. The method of claim 26 , comprising assigning a score to the one or more query term optionalization rules that are specific to the particular query term based at least on the click count and the skip count.
28. The method of claim 27 , wherein the score assigned to the one or more query term optionalization rules that are specific to the particular query term is based on a ratio of (i) the click count to (ii) the click count and the skip count.
29. The method of claim 27 , comprising:
determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy a threshold; and
removing the one or more query term optionalization rules that are specific to the particular query term from a set of query term optionalization rules that includes other query term optionalization rules that are specific to other query terms, the other query term optionalization rules indicating whether the other query terms should be made optional in revisions of search queries that include the other query terms, based on determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy the threshold.
30. The method of claim 27 , wherein the score assigned to the one or more query term optionalization rules that are specific to the particular query term satisfies:
W
1
(
click
count
)
W
1
(
click
count
)
+
W
2
(
skip
count
)
wherein W1 represents the weight associated with the click count and W2 represents the weight associated with the skip count.
31. The method of claim 26 , comprising associating a first weight with the click count and a second weight with the skip count.
32. The method of claim 25 , wherein the one or more revised search queries were generated by the search engine system using one or more query term optionalization rules that are specific to the particular query term and are associated with a particular context,
wherein the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context indicate whether the particular query term should be made optional in revisions of search queries that include the particular query term and the particular other query term,
wherein the initial search query includes the particular other query term, and
wherein adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term comprises adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context.