Progressive prefetching
The present invention relates to systems, apparatus, and methods of using usage data to determine the dependency structures of a web application, including dependency structures between follow-on objects of an initial object in a web transaction. In one embodiment, an input URL and associated dynamic response data are analyzed for such nested or dependent relationships. In further embodiments, analysis of these relationships are used to improve prefetching operations to lower overall page load times.
1. A method comprising:
observing, at a computing device, a first root request having a first set of parameters;
observing a first child request for a first child object;
identifying a correlation between the first set of parameters and the first child request;
creating, using the computing device, a prediction model that associates at least one parameter from the first set of parameters with the first child request;
observing, at the computing device, a second root request having a second set of parameters different from the first set of parameters;
determining that the second set of parameters includes a parameter value that matches the at least one parameter associated in the prediction model; and
generating, using the computing device and based on the prediction model, a prefetch hint identifying the first child object.
2. The method of claim 1 , wherein the first set of parameters comprises at least one of: a hostname, a folder path, a query parameter, a cookie value, or a header value.
3. The method of claim 1 , wherein creating the prediction model comprises:
identifying a template pattern in the first root request;
creating a bin associated with the template pattern; and
storing a mapping between the at least one parameter and the first child request in the bin.
4. The method of claim 3 , further comprising:
determining a probability score for the mapping based on subsequent web page transactions; and
updating the prediction model based on the probability score.
5. The method of claim 1 , further comprising:
tracking hits, misses, and failures associated with prefetch operations using the prediction model; and
adjusting the prediction model based on the tracked hits, misses, and failures.
6. The method of claim 1 , wherein creating the prediction model comprises:
analyzing timing data associated with receipt of the first child request relative to the first root request; and
incorporating the timing data into the prediction model to determine when to generate prefetch hints.
7. The method of claim 1 , further comprising:
identifying dynamic fields in the first child request;
determining how to synthesize values for the dynamic fields; and
including synthesis instructions in the prediction model.
8. The method of claim 1 , wherein providing the prefetch hint comprises:
transmitting the prefetch hint to a client device that generated the second root request to enable the client device to prefetch the first child object.
9. The method of claim 1 , wherein creating the prediction model comprises:
creating a mapping of the at least one parameter from the first set of parameters to the first child request;
analyzing whether parameters are consequential or inconsequential to the first child request; and
including consequential parameters in the mapping.
10. The method of claim 1 , further comprising:
sharing the prediction model with multiple proxy servers to enable distributed prefetching operations using the prediction model.
11. The method of claim 1 , wherein the computing device is a proxy server.
12. The method of claim 1 , wherein the computing device is a client device.
13. The method of claim 1 , wherein observing the first root request comprises receiving the first root request.
14. The method of claim 1 , wherein observing the first child request comprises receiving the first child request.
15. The method of claim 1 , wherein the first root request is an HTTP request.
16. The method of claim 1 , wherein the first child request is an HTTP request.
17. A system comprising:
a non-transitory computer-readable medium; and
a processor in communication with the non-transitory computer-readable medium, the processor configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:
observe a first root request having a first set of parameters;
observe a first child request for a first child object;
identify a correlation between the first set of parameters and the first child request;
create a prediction model that associates at least one parameter from the first set of parameters with the first child request;
observe a second root request having a second set of parameters different from the first set of parameters;
determine that the second set of parameters includes a parameter value that matches the at least one parameter associated in the prediction model; and
generate based on the prediction model, a prefetch hint identifying the first child object.
18. The system of claim 17 , wherein the first set of parameters comprises at least one of: a hostname, a folder path, a query parameter, a cookie value, or a header value.
19. The system of claim 17 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to create the prediction model by:
identifying a template pattern in the first root request;
creating a bin associated with the template pattern; and
storing a mapping between the at least one parameter and the first child request in the bin.
20. The system of claim 19 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:
determine a probability score for the mapping based on subsequent web page transactions; and
update the prediction model based on the probability score.
21. The system of claim 17 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:
track hits, misses, and failures associated with prefetch operations using the prediction model; and
adjust the prediction model based on the tracked hits, misses, and failures.
22. The system of claim 17 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to create the prediction model by:
analyzing timing data associated with receipt of the first child request relative to the first root request; and
incorporating the timing data into the prediction model to determine when to generate prefetch hints.
23. The system of claim 17 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:
identify dynamic fields in the first child request;
determine how to synthesize values for the dynamic fields; and
include synthesis instructions in the prediction model.
24. The system of claim 17 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to provide the prefetch hint by:
transmitting the prefetch hint to a client device that generated the second root request to enable the client device to prefetch the first child object.
25. The system of claim 17 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to create the prediction model by:
creating a mapping of the at least one parameter from the first set of parameters to the first child request;
analyzing whether parameters are consequential or inconsequential to the first child request; and
including consequential parameters in the mapping.
26. The system of claim 17 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:
share the prediction model with multiple proxy servers to enable distributed prefetching operations using the prediction model.
27. The system of claim 17 , wherein the processor is on a proxy server.
28. The system of claim 17 , wherein the processor is on a client device.
29. The system of claim 17 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to receive the first root request.
30. The system of claim 17 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to receive the first child request.
31. The system of claim 17 , wherein the first root request is an HTTP request.
32. The system of claim 17 , wherein the first child request is an HTTP request.