System and method for building a script for a web page using an existing script from a similar web page
A system and method allows users to provide portions of scripts or scripts for retrieval of information from one or more web sites of one or more businesses by demonstrating operation of the one or more web sites and identifying locations of one or more fields on each web page of the one or more web sites, and the system and method stores the scripts and uses the scripts to retrieve information from such web site or web sites for any number of users. Different portions of different scripts may be used as a single script to retrieve information from a single web site. Scripts or portions of scripts may be repaired using information from previously working scripts or portions, the web site when the script or portion worked, and the web site when the script or portion did not work.
1. A method of building at least a portion of a script that retrieves information from a website, the method comprising:
receiving, at a computing system via a network, a request from a user to retrieve information from the website;
determining, via the computing system, that no satisfactory script for retrieving the requested information from the website exists in a database;
receiving, at the computing system via the network, an indication of a page type for a first web page of the website from the user;
retrieving, via the network, the first web page via a network;
identifying, via the computing system, a candidate script element previously defined for the page type based on a second web page;
determining, via the computing system, a candidate similarity score that measures a similarity between a part of the first web page and a part of a second web page, wherein the candidate script element is configured to perform a function on the part of the second web page;
determining the candidate similarity score meets a predefined threshold; and
performing, via the candidate script element, the function on the part of the first web page based on the determination that the candidate similarity score meets the predefined threshold.
2. The method of claim 1 , wherein performing the function includes logging in to the website, navigating to a third page where the requested information is located, or retrieving the information from the website.
3. The method of claim 1 , wherein determining that no satisfactory script exists for retrieving the requested information from the website in the database includes determining that a success score for a script in the database does not meet a threshold success score.
4. The method of claim 1 , wherein determining the candidate similarity score includes:
determining that a text phrase on the first page matches a text phrase on the second page; and
adding a first amount to the candidate similarity score based on the determination that the text phrase of the first page matches the text phrase on the second page.
5. The method of claim 4 , wherein determining the candidate similarity score further includes:
determining that an individual word on the first page matches an individual word on the second page; and
adding a second amount to the candidate similarity score based on the determination that the individual word on the first page matches the individual word on the second page, wherein the second amount is less than the first amount.
6. The method of claim 5 , wherein determining the candidate similarity score further includes:
determining that an additional word on the second page is a synonym of an additional word on the second page; and
adding a third amount to the candidate similarity score based on the determination that the additional word on the second page is a synonym of the additional word on the second page.
7. The method of claim 4 , wherein the text phrase on the first page is located in a header of the first page or a title of the first page.
8. The method of claim 1 , further comprising:
identifying, via the computing system, an alternative script element previously defined for the page type based on a third web page;
determining, via the computing system, an alternative similarity score that measures a similarity between the part of the first web page and a part of a third web page, wherein the alternative script element is configured to perform the function on the part of the third web page;
determining, via the computing system, that the candidate similarity score exceeds the alternative similarity score; and
adding the candidate script element instead of the alternative script element to a script for retrieving the requested information based on the determination that the candidate similarity score exceeds the alternative similarity score.
9. A system comprising:
one or more processors; and
memory storing one or more applications that, when executed on the one or more processors, perform an operation comprising:
receiving, at a computing system via a network, a request from a user to retrieve information from the website;
determining, via the computing system, that no satisfactory script for retrieving the requested information from the website exists in a database;
receiving, at the computing system via the network, an indication of a page type for a first web page of the website from the user;
retrieving, via the network, the first web page via a network;
identifying, via the computing system, a candidate script element previously defined for the page type based on a second web page;
determining, via the computing system, a candidate similarity score that measures a similarity between a part of the first web page and a part of a second web page, wherein the candidate script element is configured to perform a function on the part of the second web page;
determining the candidate similarity score meets a predefined threshold; and
performing, via the candidate script element, the function on the part of the first web page based on the determination that the candidate similarity score meets the predefined threshold.
10. The system of claim 9 , wherein performing the function includes logging in to the website, navigating to a third page where the requested information is located, or retrieving the information from the website.
11. The system of claim 9 , wherein determining that no satisfactory script exists for retrieving the requested information from the website in the database includes determining that a success score for a script in the database does not meet a threshold success score.
12. The system of claim 9 , wherein determining the candidate similarity score includes:
determining that a text phrase on the first page matches a text phrase on the second page; and
adding a first amount to the candidate similarity score based on the determination that the text phrase of the first page matches the text phrase on the second page.
13. The system of claim 12 , wherein determining the candidate similarity score further includes:
determining that an individual word on the first page matches an individual word on the second page; and
adding a second amount to the candidate similarity score based on the determination that the individual word on the first page matches the individual word on the second page, wherein the second amount is less than the first amount.
14. The system of claim 13 , wherein determining the candidate similarity score further includes:
determining that an additional word on the second page is a synonym of an additional word on the second page; and
adding a third amount to the candidate similarity score based on the determination that the additional word on the second page is a synonym of the additional word on the second page.
15. The system of claim 12 , wherein the text phrase on the first page is located in a header of the first page or a title of the first page.
16. The system of claim 9 , further comprising:
identifying, via the computing system, an alternative script element previously defined for the page type based on a third web page;
determining, via the computing system, an alternative similarity score that measures a similarity between the part of the first web page and a part of a third web page, wherein the alternative script element is configured to perform the function on the part of the third web page;
determining, via the computing system, that the candidate similarity score exceeds the alternative similarity score; and
adding the candidate script element instead of the alternative script element to a script for retrieving the requested information based on the determination that the candidate similarity score exceeds the alternative similarity score.
17. A non-transitory computer-readable storage medium containing instructions that, when executed by one or more processors, perform an operation comprising:
receiving, at a computing system via a network, a request from a user to retrieve information from the website;
determining, via the computing system, that no satisfactory script for retrieving the requested information from the website exists in a database;
receiving, at the computing system via the network, an indication of a page type for a first web page of the website from the user;
retrieving, via the network, the first web page via a network;
identifying, via the computing system, a candidate script element previously defined for the page type based on a second web page;
determining, via the computing system, a candidate similarity score that measures a similarity between a part of the first web page and a part of a second web page, wherein the candidate script element is configured to perform a function on the part of the second web page;
determining the candidate similarity score meets a predefined threshold; and
performing, via the candidate script element, the function on the part of the first web page based on the determination that the candidate similarity score meets the predefined threshold.
18. The non-transitory computer-readable storage medium of claim 17 , wherein performing the function includes logging in to the website, navigating to a third page where the requested information is located, or retrieving the information from the website.
19. The non-transitory computer-readable storage medium of claim 17 , wherein determining the candidate similarity score includes:
determining that a text phrase on the first page matches a text phrase on the second page; and
adding a first amount to the candidate similarity score based on the determination that the text phrase of the first page matches the text phrase on the second page.
20. The non-transitory computer-readable storage medium of claim 17 , further comprising:
identifying, via the computing system, an alternative script element previously defined for the page type based on a third web page;
determining, via the computing system, an alternative similarity score that measures a similarity between the part of the first web page and a part of a third web page, wherein the alternative script element is configured to perform the function on the part of the third web page;
determining, via the computing system, that the candidate similarity score exceeds the alternative similarity score; and
adding the candidate script element instead of the alternative script element to a script for retrieving the requested information based on the determination that the candidate similarity score exceeds the alternative similarity score.