System and method for extracting content for submission to a search engine
A system and a method for automatically submitting Web pages to a search engine, which is preferably used for submitting dynamic Web pages, but may optionally be used for any type of Web page. The present invention features a gateway server for providing these Web pages to the search engine, either directly or optionally through an autonomous software search program. Optionally and more preferably, the gateway server modifies the Web page before serving it to the autonomous software search program and/or search engine.
1. A method of extracting a content from a Web page, wherein the extracted content is sufficient for a search engine to index said Web page, said method comprising:
identifying by a server a template of the Web page, wherein said template includes a non-essential code;
extracting by said server a content, other than the identified template from said Web page; separating, by said server, a link from the content of the Web page; and,
providing by said server said extracted content for receipt by said search engine.
2. A method of extracting a content from a Web page, wherein the extracted content is sufficient for a search engine to index said Web page, said method comprising:
identifying by a server a template of the Web page, wherein said template includes a non-essential code;
extracting by said server a content, other than the identified template from said Web page; producing a set of instructions for removing the template from the Web page; and,
providing by said server said extracted content for receipt by said search engine.
3. The method of claim 2 , further comprising extracting a field of data from the Web page and associating the field with an attribute defined by an information schema.
4. The method of claim 2 further comprising receiving by said search engine said extracted content for indexing.