Method and system for blending search engine results from disparate sources into one search result
View Patent ↗Provided is a method and system for blending a plurality of search engine results from a respective plurality of search sources into one search engine. A result list of entries is received from each search source. Each result list includes at least, for each entry on the list, a position of the entry, an address of a document, and a relevance score of the document. A determined transformation is applied to the relevance scores of at least a subset of the documents in at least one of the received result lists so as to generate transformed scores. Finally portions of the respective results lists are combined to produce a blended search engine result comprising a blended list of entries. The ordering of the blended list of entries is based, at least in part, on the transformed scores of the documents.
1. A method for blending a plurality of search engine results from a respective plurality of search sources into one search engine result, comprising:
receiving, from each search source, a respective result list of entries, the respective result list including at least, for each entry on the list, a position of the entry, an address of a document, and a relevance score of the document;
applying a determined transformation to the relevance scores of at least a subset of the documents in at least one of the received result lists so as to generate transformed scores; and
combining at least portions of the respective results lists to produce a blended search engine result comprising a blended list of entries, including ordering the blended list of entries, based at least in part on the transformed scores of the documents,
wherein the transformation is determined for a first search source of the plurality of search sources by:
measuring and storing click rate statistics for the first search source and for a second search source;
determining a figure of merit for each of the first and second search sources, the figure of merit determined, at least in part, by the measured and stored statistics; and
for the first search source, determining a set of coefficients based on the figures of merit for the first and second search sources, wherein the set of coefficients defines the transformation for the first search source.
2. The method of claim 1 , including determining the set of coefficients by applying a set of heuristic rules, one or more of the heuristic rules requiring that, when a ratio of the first figure of merit and the second figure of merit is within a predefined range of values, the coefficients of the transformation assume predefined coefficient values.
3. The method of claim 1 , wherein the transformation is an affine transformation, determined by two coefficients.
4. The method of claim 1 , wherein the transformation is a scaling transformation, determined by one coefficient.
5. The method of claim 4 , wherein the coefficient takes on a value in the range from 0.9 to 1.1.
6. The method of claim 1 , wherein the transformation is a shifting transformation, determined by one coefficient.
7. The method of claim 6 , wherein the coefficient takes on values from the range from −0.04 to 0.04.
8. The method of claim 1 , wherein measuring and storing click rate statistics comprises:
receiving a query from a user,
returning a blended result to the user, the blended result comprising a list of entries; recording which entry, if any, in the list of entries is selected by the user; and repeating the receiving, returning, and recording until a termination condition is reached.
9. The method of claim 8 , wherein recording includes recording whether the user selected no item from the list of entries.
10. The method of claim 8 , wherein measuring and storing click rate statistics further comprises:
recording a search source and a position associated with a selected entry in the blended result, the selected entry selected by the user; and
recording a search source and a position associated with an entry in the blended result not selected by the user.
11. The method of claim 10 , wherein determining a figure of merit for the first search source comprises:
determining, for each of the plurality of search sources, a respective plurality of numbers of impressions for a plurality of positions in the blended result;
determining, for the first search source, a first number of times the user selected an entry associated with the first search source in a position in the blended result; and
determining the figure of merit for the first search source based, at least in part, on the first number of times and the plurality of numbers of impressions.
12. A search engine system, comprising:
a plurality of search sources;
at least one interface for receiving a query and for sending search results;
at least one search engine coupled to the plurality of search source for receiving, from each search source, a respective result list of entries corresponding to the received query, the respective result list including at least, for each entry on the list, a position of the entry, an address of a document, and a relevance score relating the document to the received query;
a blending module configured to apply a determined transformation to the relevance scores of at least a subset of the entries in at least one of the received result lists so as to generate transformed scores, and further configured to combine at least portions of the respective results lists to produce a blended search engine result comprising a blended list of entries, including ordering the blended list of entries, based at least in part on the transformed scores; and
a transformation determining module configured to determine the transformation for a first search source of the plurality of search sources, the transformation determined module measuring and storing click rate statistics for the first search source and for a second search source, determining a figure of merit for each of the first and second search sources, the figure of merit determined, at least in part, by the measured and stored statistics, and, for lthe first search source, determining a set of coefficients based on the figures of merit for the first and second search sources, wherein the set of coefficients defines the transformation for the first search source.
13. The search engine system of claim 12 , wherein the at least one search engine includes the blending module.
14. The search engine system of claim 12 , wherein the at least one search engine includes the transformation determining module.
15. The search engine system of claim 12 , wherein the transformation determining module is further configured to determine the set of coefficients by applying a set of heuristic rules, one or more of the heuristic rules requiring that, when a ratio of the first figure of merit and second figure of merit is within a predefined range of values, the coefficients of the transformation assume predefined coefficient values.
16. The search engine system of claim 12 , wherein the transformation is an affine transformation, determined by two coefficients.
17. The search engine system of claim 12 , wherein the transformation is a scaling transformation, determined by one coefficient.
18. The search engine system of claim 12 , wherein the coefficient takes on a value in the range from 0.9 to 1.1.
19. The search engine system of claim 12 , wherein the transformation is a shifting transformation, determined by one coefficient.
20. The search engine system of claim 12 , wherein the coefficient takes on values from the range from −0.04 to 0.04.
21. The search engine system of claim 12 , wherein the search engine system is further configured to measure and store click rate statistics for the first search source and for a second search source, the configuration comprising:
the search engine further configured to receive a query from a user,
the search engine further configures to return a blended result to the user, the blended result comprising a list of entries;
the transformation determining module further configured to record which item, if any, in the list of entries is selected by the user;
the search engine system further configured to repeat the receiving, returning, and recording until a termination condition is reached.
22. The search engine system of claim 21 , wherein the transformation module is further configured to record whether the user selected no entry from the list of entries.
23. The search engine system of claim 21 , wherein the transformation module is further configured to:
record a search source and a position associated with a selected entry in the blended result, the selected entry selected by the user; and
record a search source and a position associated with an entry in the blended result not selected by the user.
24. The search engine system of claim 23 , wherein the transformation determining module is further configured to determine a figure of merit for the first search source by:
determine, for each of the plurality of search sources, a respective plurality of numbers of impressions for a plurality of positions in the blended result;
determine, for the first search source, a first number of times the user selected an entry associated with the first search source in a position in the blended result; and
determine the figure of merit for the first search source based, at least in part, on
the first number of times and the plurality of numbers of impressions.