Method and system for adaptive prefetching
A cache server prefetches one or more web pages from an origin server prior to those web pages being requested by a user. The cache server determines which web pages to prefetch based on a graph associated with a prefetch module associated with the cache server. The graph represents all or a portion of the web pages at the origin server using one or more nodes and one or more links connecting the nodes. Each link has an associated transaction weight and user weight. The transaction weight represents the importance of the link and associated web page to the origin server and may be used to control the prefetching of web pages by the cache server. The user weight may be used to change a priority associated with a request for a web page. The user weight and transaction weight may change based on criteria associated with the origin server.
1. A method for prefetching web pages, comprising:
retrieving a first web page at a cache server in response to a request for the first web page;
assigning a transaction weight to each of one or more second web pages linked to the retrieved first web page, the transaction weight indicating a relative importance of a particular web page by a source of the particular web page compared to other web pages at the source;
determining whether to prefetch and in what order the one or more second web pages linked to the retrieved first web page, wherein a second web page is prefetched when its transaction weight exceeds a prefetch threshold;
prefetching in order, prior to a request, those second web pages linked to the retrieved first web page whose transaction weight exceeds the prefetch threshold.
2. The method of claim 1 , wherein the source of the one or more second web pages is one or more origin servers, each origin server having a different prefetch threshold associated therewith.
3. The method of claim 1 , further comprising:
dynamically adjusting the prefetch threshold based on a processing load experienced at the cache server.
4. The method of claim 1 , further comprising:
dynamically adjusting the prefetch threshold based on a current bandwidth available to the cache server.
5. The method of claim 1 , further comprising:
increasing the prefetch threshold in response to increased traffic at the cache server.
6. The method of claim 1 , further comprising:
dynamically adjusting the prefetch threshold is based on a processing load experienced at the source.
7. The method of claim 1 , further comprising:
increasing a transaction weight for a particular page to be prefetched in response to the particular page being previously retrieved a set amount of times.
8. The method of claim 1 , further comprising:
decreasing a transaction weight for a particular page to be prefetched in response to the particular page not being retrieved over a defined time period.
9. The method of claim 1 , further comprising:
determining whether to update the transaction weight associated with a particular page in response to retrieval of the particular page.
10. A non-transitory computer readable storage medium including code for prefetching a web page, the code operable when executed to:
retrieve a first web page for a cache server in response to a request for the first web page;
assign a transaction weight to each of one or more second web pages linked to the retrieved first web page, the transaction weight indicating a relative importance of a particular web page by a source of the particular web page compared to other web pages at the source;
determine whether to prefetch and in what order the one or more second web pages linked to the retrieved first web page, wherein a second web page is prefetched when its transaction weight exceeds a prefetch threshold;
prefetch in order, prior to a request, those second web pages linked to the retrieved first web page whose transaction weight exceeds prefetch threshold.
11. The non-transitory computer readable storage medium of claim 10 , wherein the code is further operable to:
dynamically adjusting the prefetch threshold based on any of a processing load experienced at the cache server, a current bandwidth available to the cache server, and a processing load experienced at the source.
12. The non-transitory computer readable storage medium of claim 10 , wherein the code is further operable to:
increase the prefetch threshold in response to increased traffic at the cache server.
13. The non-transitory computer readable storage medium of claim 10 , wherein the code is further operable to:
increase a transaction weight for a particular page to be prefetched in response to the particular page being previously retrieved a set amount of times.
14. The non-transitory computer readable storage medium of claim 10 , wherein the code is further operable to:
decrease a transaction weight for a particular page to be prefetched in response to the particular page not being retrieved over a defined time period.
15. The non-transitory computer readable storage medium of claim 10 , wherein the code is further operable to:
determine whether to update the transaction weight associated with a particular page in response to retrieval of the particular page.
16. A system for prefetching a web page, comprising:
means for retrieving a first web page for a cache server in response to a request for the first web page;
means for assigning a transaction weight to each of one or more second web pages linked to the retrieved first web page, the transaction weight indicating a relative importance of a particular web page by a source of the particular web page compared to other web pages at the source;
means for determining whether to prefetch and in what order the one or more second web pages linked to the retrieved first web page, wherein a second web page is prefetched when its transaction weight exceeds a prefetch threshold;
means for prefetching in order, prior to a request, those second web pages linked to the retrieved first web page whose transaction weight exceeds the prefetch threshold.
17. The system of claim 16 , further comprising:
means for dynamically adjusting the prefetch threshold based on any of a processing load experienced at the cache server, a current bandwidth available to the cache server, a traffic level at the cache server, and a processing load experienced at the source.
18. The system of claim 16 , further comprising:
means for increasing a transaction weight for a particular page to be prefetched in response to the particular page being previously retrieved a set amount of times.
19. The system of claim 16 , further comprising:
means for decreasing a transaction weight for a particular page to be prefetched in response to the particular page not being retrieved over a defined time period.
20. The system of claim 16 , further comprising:
means for determining whether to update the transaction weight associated with a particular page in response to retrieval of the particular page.