Server evaluation of client-side script
View Patent ↗A facility is described for server-side evaluation of client-side script, such as to crawl and index dynamically generated links. In various embodiments, the facility provides an object that emulates a browser object. The provided object exposes a function for performing steps comprising receiving a first link and storing the received first link in a list of links to be processed. The facility receives a second link to be processed, loads a second document identified by the received second link, and assigns the provided object to a global variable so that a script associated with the loaded second document invokes the function exposed by the provided object instead of a function with a similar name that is exposed by the emulated browser object. The function with the similar name exposed by the emulated browser object handles the received first link without storing it in the list of links to be processed.
1. A method performed by a server computing device for evaluating a client-side script, comprising:
providing an object that emulates a browser object, the provided object exposing a function for performing steps comprising:
receiving a first link; and
storing the received first link in a list of links to be processed;
receiving a second link to be processed;
loading a second document identified by the received second link;
assigning the provided object to a global variable so that a script associated with the loaded second document invokes the function exposed by the provided object instead of a function with a similar name that is exposed by the emulated browser object wherein the function with the similar name exposed by the emulated browser object loads a first document referenced by the received first link upon receiving the first link;
building a document object model based on the loaded second document;
employing the built document object model to locate a script element in the loaded second document;
executing at the server computing device a client-side script defined within the script element;
detecting an attempt to change the built document object model;
receiving information relating to the attempted change to the built document object model;
determining whether the attempted change identifies a link; and
when the attempted change identifies a link, adding the identified link to a list of links to process.
2. The method of claim 1 further comprising downloading a second client-side script referenced by an attribute of the script element.
3. The method of claim 2 wherein the second client-side script referenced by the attribute of the script element is executed before the client-side script defined within the script element.
4. The method of claim 1 further comprising triggering events.
5. The method of claim 4 wherein the triggering comprises:
for each node in the document object model,
determining whether the node indicates an event handler attribute; and
invoking a user event handler indicated by the event handler attribute.
6. The method of claim 5 further comprising processing the loaded second document.
7. The method of claim 6 wherein the processing includes indexing text stored in the loaded second document.
8. The method of claim 6 wherein the processing includes adding to a list of links to process a link stored in the loaded second document, the link generated dynamically by the client-side script.
9. The method of claim 1 wherein the second link is retrieved from the list of links.
10. A system for evaluating client-side script at a server computing device, comprising:
a list of links to process, each link identifying a document;
a document object model component that builds a document object model based on the identified document, locates a script element of the identified document, detects an attempt to change the built document object model, receives information relating to the attempted change to the built document object model, determines whether the attempted change identifies a link, and when the attempted change identifies a link, adds the identified link to a list of links to process; and
a script engine component operating at the server computing device that, when a link in the list of links is processed, executes a script indicated by the script element of the document identified by a processed link causing a global variable to point to an emulating object that emulates a browser object, the emulating object providing a function with a name that is similar to a corresponding function that is provided by the emulated browser object, wherein the function provided by the emulating object stores a received link in the list of links to process even though the corresponding function that is provided by the emulated browser object handles the received link without storing it in the list of links to process.
11. The system of claim 10 further comprising a markup language sweeper component that transforms the identified document into a well-formed document.
12. The system of claim 10 wherein the script engine processes a client-side script that is written using a language that is similar to a JavaScript language.
13. The system of claim 10 wherein the received link is dynamically generated by the script.
14. The system of claim 10 wherein the emulating object emulates a window, location, navigator, cookie, or document browser object.
15. The system of claim 10 wherein the script employs the global variable as a substitute for the browser object without a change in the script.
16. A non-transitory computer-readable storage medium having computer-executable instructions that, when executed, cause a server computing system to perform a method of evaluating a client-side script, the method comprising:
providing an emulating object that emulates a browser object;
receiving from a list of links a link that is to be processed;
loading a document identified by the received link;
assigning the emulating object to an identifier so that a script identified by the loaded document invokes a function exposed by the emulating object instead of a corresponding function with a similar name that is exposed by the emulated browser object wherein the function with the similar name exposed by the emulated browser object handles a link it receives as a parameter but the function exposed by the emulating object stores in the list of links to be processed the link it receives as a parameter without handling the link;
building a document object model based on the loaded document;
employing the built document object model to locate a script element in the loaded document;
executing at the server computing device a client-side script defined within the script element;
detecting an attempt to change the built document object model;
receiving information relating to the attempted change to the built document object model;
determining whether the attempted change identifies a link; and
when the attempted change identifies a link, adding the identified link to a list of links to process.
17. The computer-readable medium of claim 16 wherein handling a link includes loading a document referenced by the link.