System and method of processing streaming XML data
View Patent ↗Embodiments include methods and systems for processing XML documents. One embodiment is a system that includes a method of efficiently processing XML documents received concurrently from a plurality of network connections in the form of streams of data. Other embodiments include systems configured to perform such processing of streamed XML documents. Other embodiments include systems and methods of efficiently performing document processing using digests for identifying XML document structure.
1. A method of incrementally processing structured documents, the method comprising:
receiving partial data of one of a plurality of structured documents, wherein each of the documents comprises a plurality of elements that define a hierarchy;
identifying which of the structured documents is associated with the partial data;
determining which of the elements of the document are included in the received partial data;
determining a position of each of the received elements in the hierarchy;
identifying a specified portion of the document, said specified portion comprising at least one of the received elements;
generating an isomorphic digest value based on the specified portion, wherein the isomorphic digest value comprises a cryptographic hash value based on the specified portion of the document;
providing, for processing, the isomorphic digest value, the received elements and the position of each of the received elements in the hierarchy; and
identifying data associated with at least one other document based on the isomorphic digest value.
2. The method of claim 1 , wherein the structured documents are received via at least one network.
3. The method of claim 1 , wherein determining the position of each of the received elements in the hierarchy comprises identifying at least one partially received element of the document.
4. The method of claim 3 , wherein the partially received element comprises a hierarchical ancestor of at least one of the received elements.
5. The method of claim 3 , wherein identifying the partially received element comprises receiving a start indicator of the partially received element without receiving a corresponding end indicator.
6. The method of claim 3 , wherein determining the position of each of the received elements in the hierarchy comprises maintaining data indicative of the at least one partially received element.
7. The method of claim 1 , further comprising:
identifying at least one namespace based on the received data; and
storing data indicative of the namespace, wherein providing the received partial data and the position of each of the received elements further comprises providing the data indicative of the namespace.
8. The method of claim 1 , wherein the specified portion of the document comprises a specified number of the elements of the document.
9. The method of claim 1 , wherein the structured document comprises an XML document.
10. The method of claim 9 , wherein each of the plurality of hierarchical elements comprises an XML element.
11. An apparatus for processing structured documents received over a network, the apparatus comprising:
a network interface configured to receive partial data of one of a plurality of structured documents via at least one network, wherein each of the documents comprises a plurality of elements that define a hierarchy;
a memory configured to store data associated with each of the documents;
a processor configured to:
identify which of the structured documents is associated with the partial data;
determine which of the elements of the document are included in the received partial data;
determine a position of each of the received elements in the hierarchy;
store the position and at least a portion of the determined elements in the memory;
identify a specified portion of the document, said specified portion comprising at least one of the received elements;
generate an isomorphic digest value based on the specified portion, wherein generating the isomorphic digest value comprises generating a cryptographic hash value based on the specified portion of the document;
process the partial data of the identified structured document based at least partly on the isomorphic digest value, the received elements and the position of each of the received elements in the hierarchy; and
identify data associated with at least one other document based on the isomorphic digest value.
12. The apparatus of claim 11 , wherein the processor is configured to determine the position of each of the received elements in the hierarchy by identifying at least one partially received element in the document.
13. The apparatus of claim 12 , wherein the partially received element comprises a hierarchical ancestor of at least one of the received elements.
14. The apparatus of claim 12 , wherein the processor is configured to maintain data indicative of the at least one partially received element.
15. The apparatus of claim 11 , wherein the processor is further configured to:
identify at least one namespace based on the received data; and
store data indicative of the namespace to the memory, wherein the processor is configured to provide the received partial data and the position of each of the received elements further by providing the data indicative of the namespace.
16. The apparatus of claim 11 , wherein the specified portion of the document comprises a specified number of the elements of the document.
17. A content processor for incrementally processing structured documents, the processor comprising:
means for receiving partial data of one of a plurality of structured documents, wherein each of the documents comprises a plurality of elements that define a hierarchy;
means for storing data associated with each of the documents;
means for processing the received partial data, wherein the processing means is configured to:
identify which of the structured documents is associated with the partial data;
determine which of the elements of the document are included in the received partial data;
determine a position of each of the received elements in the hierarchy;
store the position and at least a portion of the determined elements in the storing means; and
process the partial data of the identified structured document based at least partly on the received elements and the position of each of the received elements in the hierarchy;
identify a specified portion of the document, said specified portion comprising at least one of the received elements;
generate an isomorphic digest value based on the specified portion, wherein the isomorphic digest value comprises a cryptographic hash value based on the specified portion of the document; and
process the received portion of the document based at least partly on the isomorphic digest value.
18. The content processor of claim 17 , wherein the means for receiving partial data comprises at least one of a network interface and a storage.
19. The content processor of claim 17 , wherein the processing means is further configured to identify at least one namespace based on the received data, and wherein the storing means is further configured to store data indicative of the namespace, wherein processing the partial data is based at least partly on the data indicative of the namespace.
20. The content processor of claim 17 , wherein the processing means comprises at least one of a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a general purpose processor.