Detecting web probes versus regular traffic through a proxy including encrypted traffic
Techniques for using web probes for monitoring user experience including use of caching to prevent a surge of web probes on destination servers and for detecting web probe traffic through a proxy including where the traffic is encrypted. A method implemented by a proxy includes receiving encrypted traffic with an indicator in a header indicating a request for probe traffic; inspecting the request and a response for the probe traffic; and caching data associated with the response to in a cache.
1. A method implemented by a proxy comprising:
receiving encrypted traffic with an indicator in a header indicating a request for probe traffic;
inspecting the request and a response for the probe traffic based on the header;
caching data associated with the response in a cache formatted as a hash table, wherein each probe request is handled based on one of a plurality of states;
removing the data in the cache after a configurable time period; and
sending a probe to a destination server in response to the removal of the data in the cache.
2. The method of claim 1 , further comprising serving a response to additional web probes utilizing the data in the cache in lieu of forwarding the additional web probes to the destination server.
3. The method of claim 1 , further comprising
checking the cache for the response; and
serving the response out of the cache in lieu of forwarding the request to a destination.
4. The method of claim 1 , wherein the indicator is in a first HTTP CONNECT request.
5. The method of claim 1 , wherein the indicator is in proprietary HTTP header.
6. The method of claim 1 , wherein the web probe is one of a Hypertext Transfer Protocol (HTTP) and HTTP-Secure (HTTPS) request to a Uniform Resource Locator (URL) of the destination server.
7. The method of claim 1 , wherein the proxy is configured to not inspect the encrypted traffic.
8. A non-transitory computer-readable storage medium having computer-readable code stored thereon for programming a proxy to perform steps of:
receiving encrypted traffic with an indicator in a header indicating a request for probe traffic;
inspecting the request and a response for the probe traffic based on the header;
caching data associated with the response in a cache formatted as a hash table, wherein each probe request is handled based on one of a plurality of states;
removing the data in the cache after a configurable time period; and
sending a probe to a destination server in response to the removal of the data in the cache.
9. The non-transitory computer-readable storage medium of claim 8 , wherein the steps include serving a response to additional web probes utilizing the data in the cache in lieu of forwarding the additional web probes to the destination server.
10. The non-transitory computer-readable storage medium of claim 8 , wherein the steps include
checking the cache for the response; and
serving the response out of the cache in lieu of forwarding the request to a destination.
11. The non-transitory computer-readable storage medium of claim 8 , wherein the indicator is in a first HTTP CONNECT request.
12. The non-transitory computer-readable storage medium of claim 8 , wherein the indicator is in proprietary HTTP header.
13. The non-transitory computer-readable storage medium of claim 8 , wherein the web probe is one of a Hypertext Transfer Protocol (HTTP) and HTTP-Secure (HTTPS) request to a Uniform Resource Locator (URL) of the destination server.
14. The non-transitory computer-readable storage medium of claim 8 , wherein the proxy is configured to not inspect the encrypted traffic.
15. A proxy comprising:
one or more processors and memory comprising instructions that, when executed, cause the one or more processors to:
receive encrypted traffic with an indicator in a header indicating a request for probe traffic;
inspect the request and a response for the probe traffic based on the header;
cache data associated with the response in a cache formatted as a hash table, wherein each probe request is handled based on one of a plurality of states;
remove the data in the cache after a configurable time period; and
send a probe to a destination server in response to the removal of the data in the cache.
16. The proxy of claim 15 , wherein the instructions that, when executed, cause the one or more processors to serve a response to additional web probes utilizing the data in the cache in lieu of forwarding the additional web probes to the destination server.
17. The proxy of claim 15 , wherein the instructions that, when executed, cause the one or more processors to
check the cache for the response; and
serve the response out of the cache in lieu of forwarding the request to a destination.