System and method of accelerating processing of streaming data
Embodiments include methods and systems for processing XML documents. One embodiment is a system that includes a method of efficiently processing XML documents received concurrently from a plurality of network connections in the form of streams of data. Other embodiments include systems configured to perform such processing of streamed XML documents. Other embodiments include systems and methods of efficiently performing document processing using digests for identifying XML document structure.
1. A method of incrementally processing XML documents, the method comprising:
receiving a packet comprising data of at least one XML document, wherein each XML document comprises a plurality of elements that define a hierarchy, said hierarchy containing context information for each element relative to a position within the XML document;
identifying which of the XML documents is associated with the packet;
identifying at least one element of the XML document in the packet;
identifying a specified portion of the document, said specified portion comprising the at least one received element;
generating a digest value based on the specified portion, the digest value indicative of the hierarchy of at least the specified portion of the XML document;
deriving speculative data by performing at least one function of the processing of the XML document;
storing, in a cache indexed by the digest value, the derived speculative data, wherein the speculative data comprises data representing an XML structure of at least the specified portion of the XML document; and
processing a subsequently received XML document based at least partly on the stored data, wherein processing the subsequently received XML document further comprises:
generating a digest value based on a specified portion of the subsequently received XML document;
comparing the digest value of the specified portion of the subsequently received XML document to digest values stored in the cache;
retrieving the data corresponding to the XML structure from the cache if the digest value of the specified portion of the subsequently received XML document matches a digest value stored in the cache;
employing the data retrieved from the cache to process at least part of the subsequently XML document; and
discarding received elements of the XML file corresponding to the XML structure.
2. The method of claim 1 , wherein receiving the packet comprises receiving the packet over a network.
3. The method of claim 1 , wherein the at least one XML element comprises an element tag and a body, and wherein the data indicative of the hierarchy of the elements comprises a string that excludes that at least the body of the at least one XML document.
4. The method of claim 2 , wherein the specified portion is identified based on the size of the data indicative of the hierarchy of elements.
5. An apparatus for processing XML documents received over a network, the apparatus comprising:
a network interface configured to receive a packet of at least one XML document via at least one network, wherein each documents comprises a plurality of elements that define a hierarchy, said hierarchy containing context information for each element relative to a position within the XML document;
a memory configured to have a cache to store data associated with each of the documents;
at least one processor configured to:
identify which of the XML documents is associated with the packet;
identify at least one element of the XML document in the packet;
identify a specified portion of the document, said specified portion comprising the at least one received element;
generate a digest value based on the specified portion, wherein the digest value is indicative of the hierarchy of at least the specified portion of the XML document;
derive speculative data by performing at least one function of the processing of the XML document;
store, in the cache, indexed by the digest value, the derived speculative data, wherein the derived speculative data comprises data representing an XML structure of at least the specified portion of the XML document;
compare a digest value of a specified portion of a subsequently received XML document to digest values stored in the cache;
retrieve the data corresponding to the XML structure from the cache if the digest value of the specified portion of the subsequently received XML document matches a digest value stored in the cache;
employ the data retrieved from the cache to process at least part of the subsequently XML document; and
discard received elements of the XML file corresponding to the XML structure.
6. The apparatus of claim 5 , wherein the at least one XML element comprises an element tag and a body, and wherein the data indicative of the hierarchy of the elements comprises a string that excludes that at least the body of the at least one XML element.
7. The apparatus of claim 6 , wherein the specified portion is identified based on the size of the data indicative of the hierarchy of the elements.
8. An apparatus for incrementally processing XML documents, the apparatus comprising:
means for receiving a portion of at least one XML document via at least one network, wherein each document comprises a plurality of elements that define a hierarchy, said hierarchy containing context information for each element relative to a position within the XML document;
means for storing data associated with each of the documents;
means for processing the XML documents, said means configured to:
identify which of the XML documents is associated with the received portion;
identify at least one element of the XML document in the received portion;
identify a specified portion of the document, said specified portion comprising the at least one received element; and
generate a digest value based on the specified portion, wherein the digest value is indicative of the hierarchy of at least the specified portion of the XML document;
derive speculative data by performing at least one function of the processing of the XML document;
store, in the cache, based on the digest value, the derived speculative data, wherein the speculative data comprises data representing an XML structure of at least the specified portion of the XML document;
compare a digest value of a specified portion of a subsequently received XML document to digest values stored in the cache;
retrieve the data corresponding to the XML structure from the cache if the digest value of the specified portion of the subsequently received XML document matches a digest value stored in the cache;
employ the data retrieved from the cache to process at least part of the subsequently XML document; and
discard received elements of the XML file corresponding to the XML structure.
9. The apparatus of claim 8 , wherein the receiving means comprises at least one of a network interface and a storage element.
10. The apparatus of claim 8 , wherein the at least one XML element comprises an element tag and a body, and wherein the data indicative of the hierarchy of the elements comprises a string that excludes that at least the body of the at least one XML element.
11. The apparatus of claim 10 , wherein the specified portion is identified based on the size of the data indicative of the hierarchy of the elements.