Systems and methods for rewriting a stream of data via intermediary
A streaming rewrite method and system that can execute an efficient multiple pattern search method that parses a response in a data structure of an appliance. The method and system can avoid copying to a buffer by parsing data across a data structure to identify search patterns and phrases that may be identified by one or more actions and/or rules of an appliance or system. A parser can input one or more search patterns, and parse a body of a response or one or more packets for the search patterns. The parser can obtain pattern information about the packets and/or the response, and store this information in a database. The appliance can then perform lookups in the database for pattern information and perform rewrites in accordance with the stored pattern information. The rewritten response and/or packets can then be transmitted to a destination.
1. A method for rewriting a stream of content traversing a device intermediary to a client and a server, the method comprising:
(a) identifying, by a device intermediary to a client and a server, a plurality of patterns to match for rewriting a stream of content received by the device via a plurality of packets over a connection between the client and the server traversing the device,
(b) performing, by the device via a single parsing of content of a packet, matching of the plurality of patterns to the content of a payload of the packet of the plurality of packets;
(c) storing, by the device, results of the matching to a database associated with the connection;
(d) storing, by the device, the packet to a rewrite buffer with a predetermined size threshold for storing packets to be rewritten before transmission;
(e) performing, by the device responsive to a determination that the size of the rewrite buffer exceeds the predetermined size threshold, a rewrite on an oldest packet removed from the rewrite buffer based on results of matching performed on the oldest packet stored in the database; and
(f) transmitting, by the device, the rewritten oldest packet.
2. The method of claim 1 , wherein step (a) further comprises identifying, by the device, one of a rule or an action comprising a pattern to match for rewriting.
3. The method of claim 1 , wherein step (a) further comprises receiving, by the device, a response from the server to a request of the client via the plurality of packets, a payload of each packet of the plurality of packets comprises a portion of a body of the response.
4. The method of claim 1 , wherein step (b) further comprises storing, by the device, the packet to a data structure and performing by the device a single parsing of content of the data structure to match the plurality of patterns to the content in a single pass.
5. The method of claim 1 , wherein step (c) further comprises combining multiple patterns into a single match query of the content.
6. The method of claim 1 , wherein step (d) further comprises configuring the predetermined size threshold of the rewrite buffer to store a predetermined amount of content across a plurality of packets.
7. The method of claim 1 , wherein step (e) further comprises removing one or more oldest packets from the rewrite buffer until the size is below the predetermined size threshold.
8. The method of claim 1 , wherein step (e) further comprises looking up, by the device, in the database an offset into the packet corresponding to a matched pattern to rewrite.
9. The method of claim 1 , wherein step (e) further comprises performing, by the device, the rewrite according to an action specifying a first pattern to match, a second pattern for replacing the first pattern, and a number of bytes of a body of the response to search for the pattern.
10. The method of claim 1 , wherein step (f) further comprises rewriting each of the oldest packets as they are removed from the rewrite buffer and transmitting each rewritten oldest packet to one of the client or the server.
11. A system for rewriting a stream of content traversing a device intermediary to a client and a server, the system comprising:
a device intermediary to a client and a server received receiving a plurality of packets via a connection between the client and the server,
a packet engine of the device identifying a plurality of patterns to match for rewriting a stream of content in the plurality of packets;
a parser performing via a single parsing of content of a packet matching of the plurality of patterns to the content of a payload of the packet of the plurality of packets and storing results of matching to a database associated with the connection;
a rewrite buffer with a predetermined size threshold for storing packets to be rewritten before transmission;
a rewriter performing responsive to a determination that the size of the rewrite buffer exceeds the predetermined size threshold, a rewrite on an oldest packet removed from the rewrite buffer based on results of matching performed on the oldest packet stored in the database; and
wherein the device transmits the rewritten oldest packet.
12. The system of claim 11 , wherein the packet engine identifies one of a rule or an action comprising a pattern to match for rewriting.
13. The system of claim 11 , wherein the device receives a response from the server to a request of the client via the plurality of packets, a payload of each packet of the plurality of packets comprises a portion of a body of the response.
14. The system of claim 11 , wherein the packet engine stores the packet to a data structure and the parser performs a single parsing of content of the data structure to match the plurality of patterns to the content in a single pass.
15. The system of claim 11 , wherein the parser combines multiple patterns into a single match query of the content.
16. The system of claim 11 , wherein the predetermined size threshold of the rewrite buffer is configured to store a predetermined amount of content across a plurality of packets.
17. The system of claim 11 , wherein the rewriter removes one or more oldest packets from the rewrite buffer until the size is below the predetermined size threshold.
18. The system of claim 11 , wherein the rewriter looks up in the database an offset into the packet corresponding to a matched pattern to rewrite.
19. The system of claim 11 , wherein the rewriter performs the rewrite according to an action specifying a first pattern to match, a second pattern for replacing the first pattern, and a number of bytes of a body of the response to search for the pattern.
20. The system of claim 11 , wherein the rewriter rewrites each of the oldest packets as they are removed from the rewrite buffer and transmits each rewritten oldest packet to one of the client or the server.