Configuring web crawler to extract web page information

1. A system for configuring web crawling, comprising:

one or more processors to:

obtain a webpage comprising a plurality of nodes;

present the webpage to a user;

receive a user selection of a node in the webpage, the node comprising at least one element;

in response to the user selection of the node, present a web crawling configuration option pertaining to a web crawling action to be performed with respect to the node,

the web crawling configuration option depending at least in part on a type of an element included in the node;

receive a user input specifying the web crawling configuration options pertaining to the web crawling action to be performed with respect to the node and a user indication to specify repeatedly the web crawling action to be performed on the plurality of nodes;

specify repeatedly the web crawling action to be performed for the plurality of nodes;

store user specified web crawling configuration options;

perform the web crawling action on the plurality of nodes according to the user input;

receive another user selection of a second node in the webpage; and

receive a user input specifying an option pertaining to a web crawling action to be performed with respect to the second node; and

one or more memories coupled to the one or more processors, to provide the one or more processors with instructions.

2. The system of claim 1 , wherein the one or more processors are further to process the obtained webpage such that each node is configurable for web crawling.

3. The system of claim 1 , wherein the one or more processors are further to present a preview of the performed web crawling action according to the user input.

4. The system of claim 1 , wherein the webpage is presented to the user via a graphical user interface (GUI).

5. The system of claim 1 , wherein the web crawling action comprises text extraction.

6. The system of claim 5 , wherein the web crawling configuration options for text extraction comprise configurations for one or more of: a filter, address extraction, date extraction, extracted text name, extracted text description, and format.

7. The system of claim 1 , wherein the web crawling configuration option comprises settings associated with a node that is a link and the web crawling action comprises downloading the link.

8. The system of claim 7 , wherein downloading the link comprises obtaining a subpage; and the one or more processors are further to present a web crawling configuration option pertaining to a web crawling action to be performed with respect to one or more nodes on the subpage.

9. The system of claim 8 , the one or more processors are further to present a navigation tree of webpages, and associated web crawling configuration options with the webpage in the navigation tree of webpages.

10. The system of claim 1 , wherein the web crawling configuration option pertaining to the web crawling action comprises one or more of the following link processing settings: extracting link text, extracting alt text, and extracting link target URL.

11. The system of claim 1 , wherein the web crawling configuration option comprises a setting for downloading an image, capturing the image, or both.

12. The system of claim 1 , wherein the web crawling configuration option comprises a setting for an event associated with the node, wherein the event includes: click, double click, mouse hover, rolling wheel operation, or mouse exit.

13. The system of claim 1 , wherein the web crawling configuration option comprises a setting for a form element comprising a text box, wherein the setting comprises input text.

14. The system of claim 1 , wherein the web crawling configuration option comprises a setting for a form element comprising: a text box, a button, a radio button, or a check box.

15. The system of claim 1 , the one or more processors are further to store a locator of the node.

16. The system of claim 15 , wherein the locator of the node selected by the user is based on XPath.

17. The system of claim 1 , wherein performing the web crawling action on the node comprises downloading the webpage to be crawled.

18. The system of claim 1 , wherein performing the web crawling action on the node comprises rendering Javascript code of at least a portion of the webpage.

19. The system of claim 1 , further comprising storing results of the performed web crawling action.

20. A method for configuring web crawling, comprising:

obtaining, using one or more computer processors, a webpage comprising a plurality of nodes;

presenting the webpage to a user;

receiving a user selection of a node in the webpage, the node comprising at least one element;

in response to the user selection of the node,

presenting a web crawling configuration option pertaining to a web crawling action to be performed with respect to the node, the web crawling configuration option depending at least in part on a type of an element included in the node;

receiving a user input specifying the web crawling configuration options pertaining to the web crawling action to be performed with respect to the node and a user indication to specify repeatedly the web crawling action to be performed on the plurality of nodes;

specifying repeatedly the web crawling action to be performed for the plurality of nodes:

storing user specified web crawling configuration options;

performing the web crawling action on the plurality of nodes according to the user input;

receiving another user selection of a second node in the webpage; and

receiving a user input specifying an option pertaining to a web crawling action to be performed with respect to the second node.

21. A computer program product for configuring web crawling, the computer program product being embodied in a tangible non-transitory computer readable storage medium and comprising computer instructions for:

obtaining, using one or more computer processors, a webpage comprising a plurality of nodes;

presenting the webpage to a user;

receiving a user selection of a node in the webpage, the node comprising at least one element;

in response to the user selection of the node, presenting a web crawling configuration option pertaining to a web crawling action to be performed with respect to the node, the web crawling configuration option depending at least in part on a type of an element included in the node;

receiving a user input specifying the web crawling configuration options pertaining to the web crawling action to be performed with respect to the node and a user indication to specify repeatedly the web crawling action to be performed on the plurality of nodes;

specifying repeatedly the web crawling action to be performed for the plurality of nodes.

storing user specified web crawling configuration options;

performing the web crawling action on the plurality of nodes according to the user input;

receiving another user selection of a second node in the webpage; and

receiving a user input specifying an option pertaining to a web crawling action to be performed with respect to the second node.