Systems and methods of pipelining multiple document node streams through a query processor

1. A computer-implemented method, comprising:

at a computer having memory and a processor,

identifying a first candidate document at a first data source and a second candidate document at a second data source in response to a search request from a user, wherein the search request includes one or more keywords;

retrieving, respectively, a first set of data packets corresponding to the first candidate document from the first data source and a second set of data packets corresponding to the second candidate document from the second data source, wherein the first data source operates independently from the second data source;

generating a first node stream for the first candidate document and a second node stream for the second candidate document using the first and second sets of data packets received from the respective first and second data sources; and

repeatedly processing the first node stream and the second node stream in an alternating manner until a chunk matching the one or more keywords is identified within one of the first candidate document and the second candidate document; and

preparing the identified chunk for display to the requesting user.

2. A computer system, comprising:

memory;

one or more processors;

one or more programs stored in the memory and configured for execution by the one or more processors, the one or more programs including:

instructions for identifying a first candidate document at a first data source and a second candidate document at a second data source in response to a search request from a user, wherein the search request includes one or more keywords;

instructions for retrieving, respectively, a first set of data packets corresponding to the first candidate document from the first data source and a second set of data packets corresponding to the second candidate document from the second data source, wherein the first data source operates independently from the second data source;

instructions for generating a first node stream for the first candidate document and a second node stream for the second candidate document using the first and second sets of data packets received from the respective first and second data sources; and

instructions for repeatedly processing the first node stream and the second node stream in an alternating manner until a chunk matching the one or more keywords is identified within one of the first candidate document and the second candidate document; and

instructions for preparing the identified chunk for display to the requesting user.

3. A computer readable storage medium having stored therein instructions, which when executed by a computer system cause the computer system to:

identify a first candidate document at a first data source and a second candidate document at a second data source in response to a search request from a user, wherein the search request includes one or more keywords;

retrieve, respectively, a first set of data packets corresponding to the first candidate document from the first data source and a second set of data packets corresponding to the second candidate document from the second data source, wherein the first data source operates independently from the second data source;

generate a first node stream for the first candidate document and a second node stream for the second candidate document using the first and second sets of data packets received from the respective first and second data sources; and

repeatedly process the first node stream and the second node stream in an alternating manner until a chunk matching the one or more keywords is identified within one of the first candidate document and the second candidate document; and

prepare the identified chunk for display to the requesting user.

4. The method of claim 1 , further comprising:

submitting a document-retrieval request to the first data source; and

receiving a response from the first data source, the response including multiple data packets corresponding to the first candidate document.

5. The method of claim 1 , wherein the repeated processing of the first node stream and the second node stream in an alternating manner further includes:

after processing a last to-be-processed node in the first node stream and if no new node is inserted into the first node stream for a first predefined amount of time,

processing one or more to-be-processed nodes in the second node stream to search for the chunk within the second candidate document.

6. The method of claim 1 , wherein the first data source is different from the second data source.

7. The method of claim 1 , wherein the first data source and the second data source are two different web servers.

8. The method of claim 1 , wherein the first candidate document is an HTML web page.

9. The computer system of claim 2 , further comprising:

instructions for submitting a document-retrieval request to the first data source; and

instructions for receiving a response from the first data source, the response including multiple data packets corresponding to the first candidate document.

10. The computer system of claim 2 , wherein the instructions for repeated processing of the first node stream and the second node stream in an alternating manner further include:

instructions for processing one or more to-be-processed nodes in the second node stream to search for the chunk within the second candidate document after a last to-be-processed node in the first node stream is processed and if no new node is inserted into the first node stream for a first predefined amount of time; and

instructions for resuming processing one or more to-be-processed nodes in the first node stream to search for the chunk within the first candidate document after a last to-be-processed node in the second node stream is processed and if no new node is inserted into the second node stream for the first predefined amount of time.

11. The computer system of claim 2 , wherein the first data source is different from the second data source.

12. The computer system of claim 2 , wherein the first data source and the second data source are two different web servers.

13. The computer system of claim 2 , wherein the first candidate document is an HTML web page.

14. The computer readable storage medium of claim 3 , further comprising:

instructions for submitting a document-retrieval request to the first data source; and

instructions for receiving a response from the first data source, the response including multiple data packets corresponding to the first candidate document.

15. The computer readable storage medium of claim 3 , wherein the instructions for repeated processing of the first node stream and the second node stream in an alternating manner further include:

instructions for processing one or more to-be-processed nodes in the second node stream to search for the chunk within the second candidate document after a last to-be-processed node in the first node stream is processed and if no new node is inserted into the first node stream for a first predefined amount of time; and

instructions for resuming processing one or more to-be-processed nodes in the first node stream to search for the chunk within the first candidate document after a last to-be-processed node in the second node stream is processed and if no new node is inserted into the second node stream for the first predefined amount of time.

16. The computer readable storage medium of claim 3 , wherein the first data source is different from the second data source.

17. The computer readable storage medium of claim 3 , wherein the first data source and the second data source are two different web servers.

18. The computer readable storage medium of claim 3 , wherein the first candidate document is an HTML web page.

19. The method of claim 4 , wherein the document-retrieval request and the response are based on a communication protocol selected from the group consisting of TCP/IP and FTP.

20. The method of claim 4 , further comprising:

receiving one of the first set of data packets from the first data source;

extracting one or more nodes from the received data packet; and

inserting the one or more nodes into the first node stream.

21. The method of claim 5 , further comprising:

after processing a last to-be-processed node in the second node stream and if no new node is inserted into the second node stream for the first predefined amount of time,

resuming processing one or more to-be-processed nodes in the first node stream to search for the chunk within the first candidate document.

22. The method of claim 5 , further comprising:

stopping processing the first node stream if no new node is inserted into the first node stream for a second predefined amount of time, wherein the second amount of time is no less than the first predefined amount of time.

23. The computer system of claim 9 , wherein the document-retrieval request and the response are based on a communication protocol selected from the group consisting of TCP/IP and FTP.

24. The computer system of claim 9 , further comprising:

instructions for receiving one of the first set of data packets from the first data source;

instructions for extracting one or more nodes from the received data packet; and

instructions for inserting the one or more nodes into the first node stream.

25. The computer system of claim 10 , further comprising:

instructions for stopping processing the first node stream if no new node is inserted into the first node stream for a second predefined amount of time, wherein the second amount of time is no less than the first predefined amount of time.

26. The computer readable storage medium of claim 14 , wherein the document-retrieval request and the response are based on a communication protocol selected from the group consisting of TCP/IP and FTP.

27. The computer readable storage medium of claim 14 , further comprising:

instructions for receiving one of the first set of data packets from the first data source;

instructions for extracting one or more nodes from the received data packet; and

instructions for inserting the one or more nodes into the first node stream.

28. The computer readable storage medium of claim 15 , further comprising:

instructions for stopping processing the first node stream if no new node is inserted into the first node stream for a second predefined amount of time, wherein the second amount of time is no less than the first predefined amount of time.

29. The method of claim 20 , further comprising:

extracting a node fragment from the received data packet;

forming a node by combining the node fragment with one or more node fragments; and

inserting the formed node into the first node stream.

30. The computer system of claim 24 , further comprising:

instructions for extracting a node fragment from the received data packet;

instructions for forming a node by combining the node fragment with one or more node fragments; and

instructions for inserting the formed node into the first node stream.

31. The computer readable storage medium of claim 28 , further comprising:

instructions for extracting a node fragment from the received data packet;

instructions for forming a node by combining the node fragment with one or more node fragments; and

instructions for inserting the formed node into the first node stream.