Analysis and reporting of collected search activity data over multiple search engines
A tracking system passively tracks and records searches conducted by actual search engine users. The recorded data for each search event preferably includes the search query submitted, the search engine used, the search result item (e.g., web page) selected, the position (e.g., page number) of this item, and the user's IP address. The collected data is aggregated and analyzed to generate data regarding the search queries used to locate and access particular destinations (e.g., web pages and sites). This data may be used for various purposes, such as (1) to generate reports showing, e.g., how users reach a particular destination from different search engines and geographic locations, (2) to select content to display on a destination page, such that the content is relevant to search queries commonly used to locate that page, and (3) to update the database of one search engine to reflect searches conducted on other search engines.
1. A computer-implemented method, comprising:
collecting search activity data of search engine users, wherein the search activity data is descriptive of searches performed by said users on each of a plurality of index-type search engines, wherein each of the plurality of index-type search engines maintains its own proprietary index obtained by crawling web sites for building a unique database for generating search results, wherein each of the search engine users interacts with a search results page of the index-type search engine to view search results;
for a particular search destination, programmatically analyzing the collected search activity data by a computer system having a processor and memory to identify a plurality of search events in which a search engine user selected the particular search destination from a search result set to access the destination; and
for each of the plurality of search events, identifying from the collected search activity data, and recording in computer storage, at least the following information: (a) the search engine used, (b) the search query submitted, and (c) a position at which the destination was listed in the search result set.
2. The method of claim 1 , wherein the particular search destination is a particular web site.
3. The method of claim 1 , wherein the particular search destination is a particular web page.
4. The method of claim 3 , wherein the method comprises collecting the search activity data, at least in part, via browser-executable code embedded in said web page.
5. The method of claim 4 , wherein the browser-executable code is JavaScript code.
6. The method of claim 1 , wherein the method comprises collecting the search activity data from at least one of the following sources: browser toolbars of the search engine users; a web server access log associated with the destination; or a search engine log.
7. The method of claim 1 , wherein said position is identified as a search result page number.
8. The method of claim 1 , wherein collecting the search activity data comprises decoding HTTP referrers of a plurality of different HTTP referrer formats, each format corresponding to a different one of said plurality of index-type search engines.
9. The method of claim 1 , wherein the method further comprises using IP addresses associated with the search events to identify geographic locations of associated search engine users.
10. The method of claim 1 , further comprising analyzing said information over the plurality of search events and over the plurality of index-type search engines to generate statistical data reflective of how search engine users locate the destination.
11. The method of claim 10 , wherein analyzing said information comprises sorting said position information by search engine and by query.
12. The method of claim 10 , further comprising programmatically incorporating the statistical data into a report, said report separately specifying, for each of the plurality of index-type search engines, the search queries used to locate the destination.
13. The method of claim 12 , wherein the report further specifies, for each search engine/search query pair, statistical information about the position at which the destination was listed in a search result set.
14. The method of claim 10 , further comprising programmatically incorporating the statistical data into a report, said report separately specifying, for each of a plurality of unique search queries used to locate the destination, the index-type search engine(s) used to perform an associated search.
15. The method of claim 14 , wherein the report further specifies, for each search query/search engine pair, statistical information about the position at which the destination was listed in search results.
16. The method of claim 10 , wherein the information identified for each of the plurality of search events further includes a geographic location of an associated search engine user, and the statistical data is generated separately for each of a plurality of geographic locations.
17. The method of claim 10 , further comprising incorporating the statistical information into a ranking process used by a particular index-type search engine to rank the destination.
18. The method of claim 10 , further comprising using the statistical information to select advertising to display on said destination.
19. The method of claim 1 , further comprising automatically generating a search engine comparison report using said information, said search engine comparison report separately indicating, for each of the plurality of index-type search engines, the pluralities of different search queries used to arrive at the destination via the respective search engine.
20. A computer-readable medium having stored thereon a computer program that, when executed by a computer system, causes the computer system to perform the method of claim 1 .
21. The method of claim 1 , further comprising automatically generating an interactive search engine comparison report using said information, said search engine comparison report separately indicating, for each of the plurality of index-type search engines, the pluralities of different search queries used to arrive at the destination via the respective search engine;
wherein via interaction with a user of the interactive search engine comparison report, a data range of the report can be constrained by one or more of the following:
a time limit;
a type of industry;
search term ranking;
a limit on a geographical range comprising at least one of country, state, territory, city, or zip code; or
identification of the index-type search engine; and
displaying applicable constraints in the interactive search engine comparison report.
22. A computer system, comprising:
a data repository that stores search activity data of search engine users, said search activity data encompassing searches conducted by search engine users over a plurality of index-type search engines, wherein each of the plurality of index-type search engines maintains its own proprietary index obtained by crawling web sites for building a unique database for generating search results, wherein each of the search engine users interacts with a search results page of the index-type search engine to view search results; and
an analysis component stored on a computer-readable medium, wherein the analysis component analyzes the search activity data stored in the data repository to generate statistical data regarding search queries used over said plurality of index-type search engines to locate and access a particular destination, wherein the analysis component comprises computer hardware.
23. The computer system of claim 22 , wherein the analysis component generates separate statistical data for each of the plurality of index-type search engines.
24. The computer system of claim 22 , wherein the statistical data includes, for a particular search engine and search query, information about a position at which the index-type search engine displayed the destination in search results for said query.
25. The computer system of claim 22 , further comprising a report generator stored on a computer-readable medium, wherein the report generator incorporates the statistical data into a search engine comparison report, said search engine comparison report identifying differences between how users locate the destination using different search engines.
26. The computer system of claim 22 , further comprising a content selection system stored on a computer-readable medium, wherein the content selection system takes the statistical data into consideration in selecting content to display on said destination.
27. The computer system of claim 22 , further comprising a search engine system stored on a computer-readable medium, wherein the search engine system takes the statistical data into consideration in ranking the destination for display in said search results.
28. The system of claim 22 , further comprising a report generator stored on a computer-readable medium, wherein the report generator incorporates the statistical data into an interactive search engine comparison report, said search engine comparison report identifying differences between how users locate the destination using different search engines;
wherein via interaction with a user of the interactive search engine comparison report, a data range of the report can be constrained by one or more of the following:
a time limit;
a type of industry;
search term ranking;
a limit on a geographical range comprising at least one of country, state, territory, city, or zip code; or
identification of the index-type search engine; and
displaying applicable constraints in the interactive search engine comparison report.