Adaptive learning periods in HTML streaming
A method of speeding up the delivery of a dynamic webpage is disclosed. A plurality of responses to a plurality of requests for a webpage is received, the webpage including dynamic components. The plurality of responses is compared to identify common and dynamic components across the plurality of requests. A cached stub is dynamically adapted based on the comparison within a learning window, the cached stub including the identified common components and placeholders for portions of the identified dynamic components. The dynamically adapted cached stub is sent in response to at least some of the plurality of requests for the webpage. A frequency of reloading of the webpage is monitored, wherein a reloading of the webpage is triggered by a detection of the dynamically adapted cached stub having one of a plurality of types of error. The learning window is adjusted based on a frequency of reloading of the webpage.
1. A method of speeding up the delivery of a dynamic webpage by a proxy server to a client device, comprising:
receiving a plurality of responses to a plurality of requests for a webpage, the webpage including one or more dynamic components;
comparing the plurality of responses to identify common components and dynamic components across the plurality of requests;
dynamically updating a cached stub based on the comparison within a learning period, the cached stub including the identified common components and placeholders for portions of the identified dynamic components;
sending the dynamically updated cached stub in response to at least some of the plurality of requests for the webpage;
monitoring occurrences of reloading of the webpage wherein monitoring occurrences of reloading of the webpage comprises monitoring a frequency of occurrence of reloading of the webpage, comprising:
monitoring a first frequency of occurrence of a first type of reloading of the webpage;
monitoring a second frequency of occurrence of a second type of reloading of the webpage;
determining that the first type of reloading of the webpage is triggered by the proxy server;
determining that the second type of reloading of the webpage is triggered by the client device;
adjusting the learning period based on the monitored occurrences of reloading of the webpage, and wherein adjusting the learning period based on the monitored occurrences of reloading of the webpage comprises:
increasing the learning period in event that the monitored frequency of occurrence of reloading is above a first predetermined threshold comprising in the event that the monitored first frequency of occurrence of the first type of reloading is above the first predetermined threshold, increasing the learning period, wherein increasing the learning period comprises increasing the learning period in a plurality of phases, wherein during each successive phase the learning period is multiplied by a corresponding factor, and wherein the increasing of the learning period is continued until the monitored first frequency of occurrence of the first type of reloading of the webpage is below a second predetermined threshold; and
in the event that the monitored second frequency of occurrence of the second type of reloading is above a third predetermined threshold, disabling the sending of the dynamically updated cached stub and disabling the adjusting of the learning period for at least a period of time.
2. A proxy server for speeding up the delivery of a dynamic webpage to a client device, comprising:
a processor configured to:
receive a plurality of responses to a plurality of requests for a webpage, the webpage including one or more dynamic components;
compare the plurality of responses to identify common components and dynamic components across the plurality of requests;
dynamically update a cached stub based on the comparison within a learning period, the cached stub including the identified common components and placeholders for portions of the identified dynamic components;
send the dynamically updated cached stub in response to at least some of the plurality of requests for the webpage;
monitor occurrences of reloading of the webpage wherein monitoring occurrences of reloading of the webpage comprises monitoring a frequency of occurrence of reloading of the webpage comprising:
monitoring a first frequency of occurrence of a first type of reloading of the webpage;
monitoring a second frequency of occurrence of a second type of reloading of the webpage;
determining that the first type of reloading of the webpage is triggered by the proxy server;
determining that the second type of reloading of the webpage is triggered by the client device;
adjust the learning period based on the monitored occurrences of reloading of the webpage, and wherein adjusting the learning period based on the monitored occurrences of reloading of the webpage comprises:
increasing the learning period in event that the monitored frequency of occurrence of reloading is above a first predetermined threshold, comprising in the event that the monitored first frequency of occurrence of the first type of reloading is above the first predetermined threshold, increasing the learning period, wherein increasing the learning period comprises increasing the learning period in a plurality of phases, wherein during each successive phase the learning period is multiplied by a corresponding factor, and wherein the increasing of the learning period is continued until the monitored first frequency of occurrence of the first type of reloading of the webpage is below a second predetermined threshold; and
in the event that the monitored second frequency of occurrence of the second type of reloading is above a third predetermined threshold, disabling the sending of the dynamically updated cached stub and disabling the adjusting of the learning period for at least a period of time; and
a memory coupled to the processor and configured to provide the processor with instructions.
3. A computer program product for speeding up the delivery of a dynamic webpage by a proxy server to a client device, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
receiving a plurality of responses to a plurality of requests for a webpage, the webpage including one or more dynamic components;
comparing the plurality of responses to identify common components and dynamic components across the plurality of requests;
dynamically updating a cached stub based on the comparison within a learning period, the cached stub including the identified common components and placeholders for portions of the identified dynamic components;
sending the dynamically updated cached stub in response to at least some of the plurality of requests for the webpage;
monitoring occurrences of reloading of the webpage wherein monitoring occurrences of reloading of the webpage comprises monitoring a frequency of occurrence of reloading of the webpage, comprising:
monitoring a first frequency of occurrence of a first type of reloading of the webpage;
monitoring a second frequency of occurrence of a second type of reloading of the webpage;
determining that the first type of reloading of the webpage is triggered by the proxy server;
determining that the second type of reloading of the webpage is triggered by the client device;
adjusting the learning period based on the monitored occurrences of reloading of the webpage, and wherein adjusting the learning period based on the monitored occurrences of reloading of the webpage comprises:
increasing the learning period in event that the monitored frequency of occurrence of reloading is above a first predetermined threshold, comprising in the event that the monitored first frequency of occurrence of the first type of reloading is above the first predetermined threshold, increasing the learning period, wherein increasing the learning period comprises increasing the learning period in a plurality of phases, wherein during each successive phase the learning period is multiplied by a corresponding factor, and wherein the increasing of the learning period is continued until the monitored first frequency of occurrence of the first type of reloading of the webpage is below a second predetermined threshold; and
in the event that the monitored second frequency of occurrence of the second type of reloading is above a third predetermined threshold, disabling the sending of the dynamically updated cached stub and disabling the adjusting of the learning period for at least a period of time.
4. The method of claim 1 , further comprising injecting a script in the dynamically updated cached stub, the script intercepting any access to one of the placeholders, and wherein the script causes a reloading of the webpage in response to an interception of an access to the placeholder.
5. The method of claim 1 , further comprising:
determining a difference between the dynamically updated cached stub and one of the plurality of responses; and
sending an instruction to cause a reloading of the webpage to correct the difference.
6. The proxy server of claim 2 , further comprising injecting a script in the dynamically updated cached stub, the script intercepting any access to one of the placeholders, and wherein the script causes a reloading of the webpage in response to an interception of an access to the placeholder.
7. The proxy server of claim 2 , further comprising:
determining a difference between the dynamically updated cached stub and one of the plurality of responses; and
sending an instruction to cause a reloading of the webpage to correct the difference.
8. The computer program product of claim 3 , further comprising injecting a script in the dynamically updated cached stub, the script intercepting any access to one of the placeholders, and wherein the script causes a reloading of the webpage in response to an interception of an access to the placeholder.
9. The computer program product of claim 3 , further comprising:
determining a difference between the dynamically updated cached stub and one of the plurality of responses; and
sending an instruction to cause a reloading of the webpage to correct the difference.