Progressive prefetching
The present invention relates to systems, apparatus, and methods of using usage data to determine the dependency structures of a web application, including dependency structures between follow-on objects of an initial object in a web transaction. In one embodiment, an input URL and associated dynamic response data are analyzed for such nested or dependent relationships. In further embodiments, analysis of these relationships are used to improve prefetching operations to lower overall page load times.
1. A method comprising:
receiving, using a computing device, web page transaction data for a web page transaction comprising:
a root request for a web page, and resulting therefrom:
a root response;
a child request for a child object; and
a child response;
wherein the web page transaction comprises a prior loading of the web page by a first client browser, and
wherein the web page transaction data comprises timing data indicating a relative timing of a plurality of web page transaction events during the prior loading of the web page, the timing data comprising a timing of the use of the child object by the first client browser relative to one of the plurality of web page transaction events;
determining, using the computing device, a page impact on the web page transaction of the child object based on the web page transaction data, the page impact based on a use of the child object during the web page transaction with respect to a web page transaction event;
determining, using the computing device, a caching priority based on the page impact; and
providing, using the computing device, the caching priority to enable a determination to cache the child object if the caching priority indicates the child object should be cached.
2. The method of claim 1 , wherein one of the plurality of web page transaction events is an initiation of the web page transaction by the first client browser and another of the plurality of web page transaction events is use of the child object by the first client browser.
3. The method of claim 1 , wherein one of the plurality of web page transaction events is the child request by the first client browser and another of the plurality of web page transaction events is receipt of the child response by the first client browser.
4. The method of claim 3 , wherein another of the plurality of web page transaction events is receipt of a first byte of the child response by the first client browser.
5. The method of claim 1 , wherein one of the plurality of web page transaction events is starting, by the first client browser, a rendering of the web page.
6. The method of claim 1 , wherein one of the plurality of web page transaction events is use of the child object by the first client browser and another of the plurality of web page transaction events is use of another child object of the web page by the first client browser.
7. The method of claim 1 , wherein one of the plurality of web page transaction events is a navigation event by the first client browser and another of the plurality of web page transaction events is use of the child object by the first client browser.
8. The method of claim 1 , wherein one of the plurality of web page transaction events is completion of the prior loading of the web page by the first client browser.
9. The method of claim 1 , wherein one of the plurality of web page transaction events is the root request.
10. The method of claim 1 , wherein:
the computing device is a web optimization server, and
the receiving web page transaction data comprises receiving the web page transaction data from the first client browser.
11. The method of claim 1 , wherein the providing the caching priority comprises providing the caching priority to a second client browser.
12. The method of claim 11 , wherein the caching priority enables the second client browser to make the determination to cache the child object.
13. The method of claim 11 , wherein the caching priority enables a cache external to the computing device and the first and second client browsers to make the determination to cache the child object.
14. The method of claim 1 , wherein the page impact is based on a timing of the use of the child object by the first client browser with respect to at least one of the web page transaction events during the loading of the web page.
15. The method of claim 1 , wherein the web page transaction data further comprises a probability of the first client browser or another client browser using the child object during a future loading of the web page.
16. The method of claim 1 , wherein determining the page impact is further based on an expected impact associated with caching the child object.
17. The method of claim 1 , wherein determining the page impact is further based on an impact from storing the child object in a dynamic content delivery network.
18. The method of claim 1 , wherein determining the page impact is further based on a predicted future page load frequency.
19. The method of claim 1 , wherein determining the page impact is further based on server responsiveness of a server associated with the child object.
20. The method of claim 1 , wherein the page impact is a shared page impact, and wherein determining the shared page impact is based on a sum of a plurality of page impacts of the child object over a predetermined time period.
21. The method of claim 1 , wherein determining the page impact is further based on a second page impact, the second page impact based on a use of the child object during a second web page transaction with respect to a web page transaction event, the second web page transaction for a second web page different than the web page.
22. The method of claim 1 , wherein:
the computing device is an internet optimization server.
23. A system comprising:
a non-transitory computer-readable medium; and
a processor in communication with the non-transitory computer-readable medium, the processor configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:
receive web page transaction data for a web page transaction comprising:
a root request for a web page, and resulting therefrom:
a root response;
a child request for a child object; and
a child response;
wherein the web page transaction comprises a prior loading of the web page by a first client browser, and
wherein the web page transaction data comprises timing data indicating a relative timing of a plurality of web page transaction events during the prior loading of the web page, the timing data comprising a timing of the use of the child object by the first client browser relative to one of the plurality of web page transaction events;
determine a page impact on the web page transaction of the child object based on the web page transaction data, the page impact based on a use of the child object during the web page transaction with respect to a web page transaction event;
determine a caching priority based on the page impact; and
provide the caching priority to enable a determination to cache the child object if the caching priority indicates the child object should be cached.
24. The system of claim 23 , wherein one of the plurality of web page transaction events is an initiation of the web page transaction by the first client browser and another of the plurality of web page transaction events is use of the child object by the first client browser.
25. The system of claim 23 , wherein one of the plurality of web page transaction events is the child request by the first client browser and another of the plurality of web page transaction events is receipt of the child response by the first client browser.
26. The system of claim 25 , wherein another of the plurality of web page transaction events is receipt of a first byte of the child response by the first client browser.
27. The system of claim 23 , wherein one of the plurality of web page transaction events is starting, by the first client browser, a rendering of the web page.
28. The system of claim 23 , wherein one of the plurality of web page transaction events is use of the child object by the first client browser and another of the plurality of web page transaction events is use of another child object of the web page by the first client browser.
29. The system of claim 23 , wherein one of the plurality of web page transaction events is a navigation event by the first client browser and another of the plurality of web page transaction events is use of the child object by the first client browser.
30. The system of claim 23 , wherein one of the plurality of web page transaction events is completion of the prior loading of the web page by the first client browser.
31. The system of claim 23 , wherein one of the plurality of web page transaction events is the root request.
32. The system of claim 23 , wherein:
the system is a web optimization server, and
the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to receive the web page transaction data from the first client browser.
33. The system of claim 19 , the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to provide the caching priority to a second client browser.
34. The system of claim 30 , wherein the caching priority enables the second client browser to make the determination to cache the child object.
35. The system of claim 30 , wherein the caching priority enables a cache external to the system and the first and second client browsers to make the determination to cache the child object.
36. The system of claim 23 , wherein the page impact is based on a timing of the use of the child object by the first client browser with respect to at least one of the web page transaction events during the loading of the web page.
37. The system of claim 23 , wherein the web page transaction data further comprises a probability of the first client browser or another client browser using the child object during a future loading of the web page.
38. The system of claim 19 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to determine the page impact further based on an expected impact associated with caching the child object.
39. The system of claim 19 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to determine the page impact further based on an impact from storing the child object in a dynamic content delivery network.
40. The system of claim 19 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to determine the page impact further based on a predicted future page load frequency.
41. The system of claim 19 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to determine the page impact further based on server responsiveness of a server associated with the child object.
42. The system of claim 23 , wherein the page impact is a shared page impact, and wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to determine the shared page impact based on a sum of a plurality of page impacts of the child object over a predetermined time period.
43. The system of claim 23 wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to determine the page impact further based on a second page impact, the second page impact based on a use of the child object during a second web page transaction with respect to a web page transaction event, the second web page transaction for a second web page different than the web page.
44. The system of claim 23 , wherein:
the system is an internet optimization server.