METHOD AND SYSTEM FOR AN AGGREGATE WEB SITE SEARCH DATABASE
Signature schema documents may be pre-defined using a query language to provide instructions for application by an engine to extract data from web pages of respective web sites. For a particular web page, signature schema instructions identify a web page family for the web page and extract desired data from the web page in accordance with its web page family. The instructions use signatures previously identified within web pages of the same family to distinguish the web page family from others of the web site and to distinguish the desired data from other data for the web page family. A server may make one or more requests to obtain web pages from various web sites and apply respective signature schemas maintained in a repository coupled to the engine. Extracted data can be stored to an aggregate database.
1 . A method of aggregating web site data from one or more web sites, the method comprising:
sending a page request to a web site selected from the one or more web sites;
receiving the requested web page from the selected web site;
retrieving signature schema associated with the requested web page wherein the signature scheme identifies data fields within the requested web page;
applying signature schema to the requested web page to extract data from the requested web page; and
storing extracted data to an aggregate database, wherein the aggregate database comprises data extracted from the one or more web sites.
2 . The method of claim 1 further comprising:
receiving a search query from a client machine for data stored in the aggregate database;
generating a database query based upon the received search query; and
retrieving data from the aggregate database defined by the database query.
3 . The method of claim 2 further comprising:
generating a retrieved data web page for the client machine using the retrieved data; and
providing the retrieved data web page to the client machine.
4 . The method of claim 2 wherein the search query is generated by a push application on the client machine and wherein the retrieved data is sent to the push application.
5 . The method of claim 1 wherein the data is stored in a relational database.
6 . The method of claim 5 wherein the database query is a Structured Query Language (SQL) query.
7 . The method of claim 1 wherein the signature schema is retrieved from a repository of one ore more signature schemas, wherein each schema is defined for each of the one or more web sites.
8 . The method of claim 1 wherein the signature schema comprises eXtensible Markup Language (XML) documents comprising query language for extracting data from the requested web page.
9 . The method of claim 1 wherein the aggregate database comprises a category table identifying a hypertext transport protocol (HTTP) link and a product category and wherein the aggregate database further comprises a product table identifying a product specific HTTP link, product information and pricing for each identified product category.
10 . The method of claim 1 wherein the aggregated database is stored on the client machine.
11 . The method of claim 1 where in the client machine is a wireless device.
12 . A system for aggregating web site data from one or more web sites, the system comprising:
at least one computing device comprising a processor and a memory coupled thereto, said memory storing instructions and data for configuring the processor to:
send a page request to a web site selected from the one or more web sites;
receive a web page from the selected web site based upon the sent page request;
retrieve signature schema associated with the requested web page;
apply signature schema to the requested web page data to extract data identified by the signature schema; and
store extracted data to an aggregate database comprising data extracted from the one ore more web sites.
13 . The system of claim 12 wherein the instructions for configuring the processor are further configured to:
receive a search request for data from the aggregate database from a client machine;
generate a database query based upon the received search request; and
retrieve data from the database defined by the database query.
14 . The system of claim 12 wherein the signature schema is retrieved from a repository of one ore more signature schemas wherein each schema is defined for each of the one or more web sites.
15 . The system of claim 12 wherein the signature schema are eXtensible Markup Language (XML) documents comprises query language for extracting data the requested web page.
16 . The system of claim 12 wherein the aggregate database comprises a category table identifying a hypertext transport protocol (HTTP) link and a product category and wherein the aggregate database further comprises a product table identifying a product specific HTTP link, product information and pricing for each identified product category.
17 . The system of claim 12 wherein the data is stored in a relational database.
18 . The system of claim 17 wherein the query is a Structured Query Language (SQL) query.
19 . The system of claim 12 wherein the processor is further configured to:
generate a retrieved data web page for the client machine using the retrieved data; and
provide the retrieved data web page to the client machine.
20 . The system of claim 12 where in the search query is generated by a push application on the client machine and the retrieved data is sent to the push application.
21 . The system of claim 12 wherein the aggregated data base is stored on the client machine.
22 . The system of claim 21 where in the client machine is a wireless device.
23 . A computer program product storing computer readable instructions which when executed by a computer processor configure the processor for:
sending a page request to a web site selected from the one or more web sites;
receiving the requested web page from the selected web site;
retrieving signature schema associated with the requested web page wherein the signature scheme identifies data fields within the requested web page;
applying signature schema to the requested web page to extract data from the requested web page; and
storing extracted data to an aggregate database, wherein the aggregate database comprises data extracted from the one or more web sites.
24 . A method of aggregating web site data from one or more web sites, the method comprising:
sending a page request to a web site selected from the one or more web sites;
receiving the requested web page from the selected web site;
retrieving signature schema associated with the requested web page wherein the signature scheme identifies data fields within the web and wherein the signature schema are extensible Markup Language (XML) documents comprising query language for extracting data from the requested web page;
applying signature schema to the received web page to extract data from the requested web page;
storing extracted data to an aggregate database, wherein the aggregate database comprises data extracted from the one or more web sites;
receiving a search query from a client machine for data stored in the aggregate database;
generating a database query based upon the received search query; and
retrieving data from the aggregate database defined by the query.
25 . The method of claim 25 where in the client machine is a wireless device.