Cached web probes for monitoring user experience
Techniques for using web probes for monitoring user experience including use of caching to prevent a surge of web probes on destination servers and for detecting web probe traffic through a proxy including where the traffic is encrypted. A method implemented by a proxy includes receiving a response to a first web probe to a destination server; caching data associated with the response to the first web probe in a cache; receiving a request for a second web probe to the destination server; and serving a response to the second web probe utilizing the data in the cache in lieu of forwarding the second web probe to the destination server.
1. A method implemented by a proxy comprising:
receiving a response to a first web probe to a destination server;
recording times associated with receiving the response to the first web probe;
caching data associated with the response to the first web probe in a cache,
wherein the data includes unique entries created based on request properties, and policies that govern how further requests would be forwarded to the destination server;
receiving a request for a second web probe to the destination server;
serving a response to the second web probe utilizing the data in the cache in lieu of forwarding the second web probe to the destination server, wherein the serving the response utilizes the recorded times and policies to simulate a real response, thereby matching metrics of served responses to that of real responses; and
applying all policies to the response served utilizing the data in the cache that are applied to regular traffic.
2. The method of claim 1 , further comprising
removing the data in the cache after a configurable time period.
3. The method of claim 1 , further comprising
receiving a request for the first web probe prior to the response; and
sending the first web probe to the destination server prior to the response.
4. The method of claim 1 , further comprising
receiving a third web probe while a response to a fourth web probe is outstanding;
queuing the third web probe until the response to the fourth web probe is cached;
recording the wait time in the queue; and
deducting the wait time from Time To First Byte (TTFB) and Time To Last Byte (TTLB) calculations.
5. The method of claim 1 , wherein the response to the second web probe includes truncated data padded with zeros to make sure that the time to transfer the response is factored.
6. The method of claim 1 , wherein the web probe is one of a Hypertext Transfer Protocol (HTTP) and HTTP-Secure (HTTPS) request to a Uniform Resource Locator (URL) of the destination server.
7. The method of claim 1 , wherein the data in the cache is stored as a hash table with a key being an identifier for a tenant plus a hash of the first web probe and a value is results.
8. A non-transitory computer-readable storage medium having computer-readable code stored thereon for programming a proxy to perform steps of:
receiving a response to a first web probe to a destination server;
recording times associated with receiving the response to the first web probe;
caching data associated with the response to the first web probe in a cache,
wherein the data includes unique entries created based on request properties, and policies that govern how further requests would be forwarded to the destination server;
receiving a request for a second web probe to the destination server;
serving a response to the second web probe utilizing the data in the cache in lieu of forwarding the second web probe to the destination server, wherein the serving the response utilizes the recorded times and policies to simulate a real response, thereby matching metrics of served responses to that of real responses; and
applying all policies to the response served utilizing the data in the cache that are applied to regular traffic.
9. The non-transitory computer-readable storage medium of claim 8 , wherein the steps include
removing the data in the cache after a configurable time period.
10. The non-transitory computer-readable storage medium of claim 8 , wherein the steps include
receiving a request for the first web probe prior to the response; and
sending the first web probe to the destination server prior to the response.
11. The non-transitory computer-readable storage medium of claim 8 , wherein the steps include
receiving a third web probe while a response to a fourth web probe is outstanding;
queuing the third web probe until the response to the fourth web probe is cached;
recording the wait time in the queue; and
deducting the wait time from Time To First Byte (TTFB) and Time To Last Byte (TTLB) calculations.
12. The non-transitory computer-readable storage medium of claim 8 , wherein the response to the second web probe includes truncated data padded with zeros to make sure that the time to transfer the response is factored.
13. The non-transitory computer-readable storage medium of claim 8 , wherein the web probe is one of a Hypertext Transfer Protocol (HTTP) and HTTP-Secure (HTTPS) request to a Uniform Resource Locator (URL) of the destination server.
14. The non-transitory computer-readable storage medium of claim 8 , wherein the data in the cache is stored as a hash table with a key being an identifier for a tenant plus a hash of the first web probe and a value is results.
15. A proxy comprising:
one or more processors and memory comprising instructions that, when executed, cause the one or more processors to
receive a response to a first web probe to a destination server;
record times associated with receiving the response to the first web probe;
cache data associated with the response to the first web probe in a cache,
wherein the data includes unique entries created based on request properties, and policies that govern how further requests would be forwarded to the destination server;
receive a request for a second web probe to the destination server;
serve a response to the second web probe utilizing the data in the cache in lieu of forwarding the second web probe to the destination server, wherein the serving the response utilizes the recorded times and policies to simulate a real response, thereby matching metrics of served responses to that of real responses; and
apply all policies to the response served utilizing the data in the cache that are applied to regular traffic.
16. The proxy of claim 15 , wherein the instructions that, when executed, cause the one or more processors to
remove the data in the cache after a configurable time period.
17. The proxy of claim 15 , wherein the instructions that, when executed, cause the one or more processors to
receive a request for the first web probe prior to the response; and
send the first web probe to the destination server prior to the response.
18. The proxy of claim 15 , wherein the instructions that, when executed, cause the one or more processors to
receive a third web probe while a response to a fourth web probe is outstanding;
queue the third web probe until the response to the fourth web probe is cached;
record the wait time in the queue; and
deduct the wait time from Time To First Byte (TTFB) and Time To Last Byte (TTLB) calculations.