Intelligent dataflow-based service discovery and analysis
The disclosed embodiments are directed toward monitoring and classifying encrypted network traffic. In one embodiment, a method is disclosed comprising intercepting an encrypted network request, the network request transmitted by a client device to a network endpoint; identifying a network service associated with the network endpoint based on unencrypted properties of the encrypted network request; identifying, based on the encrypted network request and a series of subsequent network requests issued by the client device, an action taken by the client device, the action comprising an activity performed during a session established with the network service; and updating a catalog of network interactions using the network service and the action.
1. A method comprising:
intercepting an encrypted network request, the encrypted network request transmitted by a client device to a network endpoint to establish a secure session;
identifying a network service associated with the network endpoint based on unencrypted properties of the encrypted network request, the encrypted network request comprising an encrypted portion and a plaintext portion, the unencrypted properties determined based on the plaintext portion;
identifying an activity performed during the secure session by:
capturing a series of subsequent encrypted network requests issued by the client device to the network endpoint during a transaction burst occurring during the secure session,
generating a fingerprint based on the series of subsequent encrypted network requests, the fingerprint comprising at least one aggregated feature computed using properties of the series of subsequent encrypted network requests,
inputting the fingerprint and an identification of the network service to a predictive model, the predictive model trained using transaction bursts by extracting one or more transaction properties from the transaction burst, assigning labels to the one or more transaction properties, a given label comprising one or more of a network service and an action, and training the predictive model with labeled data, and
using an output of the predictive model as the activity; and
updating a catalog of network interactions using the network service and the activity;
extracting a property from the encrypted network request; and
updating a mapping using the property and the network service.
2. The method of claim 1 , the identifying the network service comprising extracting a host and domain from a plaintext portion of a TLS handshake.
3. The method of claim 2 , the extracting the host and domain from the plaintext portion of the TLS handshake comprising extracting the host and domain from a sever name identification extension field of a client hello issued during the TLS handshake.
4. The method of claim 1 , the one or more transaction properties comprising a property selected from the group consisting of: a transmission control protocol (TCP) port; an Internet Protocol (IP) address space; a size of a datagram; a response time; a number of requests in the transaction burst; and a network route trace.
5. The method of claim 1 , further comprising:
extracting an IP address from encrypted network request; and
updating a mapping using the IP address and the network service, the mapping comprising a mapping of IP addresses to network services.
6. The method of claim 1 , wherein the property is selected from the group consisting of response size, response time, network route, and uniform resource identifier (URI) header length.
7. A non-transitory computer readable storage medium for tangibly storing computer program instructions capable of being executed by a computer processor, the computer program instructions defining steps of:
intercepting an encrypted network request, the encrypted network request transmitted by a client device to a network endpoint to establish a secure session;
identifying a network service associated with the network endpoint based on unencrypted properties of the encrypted network request, the encrypted network request comprising an encrypted portion and a plaintext portion, the unencrypted properties determined based on the plaintext portion;
identifying an activity performed during the secure session by:
capturing a series of subsequent encrypted network requests issued by the client device to the network endpoint during a transaction burst occurring during the secure session,
generating a fingerprint based on the series of subsequent encrypted network requests, the fingerprint comprising at least one aggregated feature computed using properties of the series of subsequent encrypted network requests,
inputting the fingerprint and an identification of the network service to a predictive model, the predictive model trained using transaction bursts by extracting one or more transaction properties from the transaction burst, assigning labels to the one or more transaction properties, a given label comprising one or more of a network service and an action, and training the predictive model with labeled data, and
using an output of the predictive model as the activity; and
updating a catalog of network interactions using the network service and the activity;
extracting a property from the encrypted network request; and
updating a mapping using the property and the network service.
8. The non-transitory computer readable storage medium of claim 7 , the identifying the network service comprising extracting a host and domain from a plaintext portion of a TLS handshake.
9. The non-transitory computer readable storage medium of claim 8 , the extracting the host and domain from the plaintext portion of the TLS handshake comprising extracting the host and domain from a sever name identification extension field of a client hello issued during the TLS handshake.
10. The non-transitory computer readable storage medium of claim 7 , the one or more transaction properties comprising a property selected from the group consisting of: a transmission control protocol (TCP) port; an Internet Protocol (IP) address space; a size of a datagram; a response time; a number of requests in the transaction burst; and a network route trace.
11. The non-transitory computer readable storage medium of claim 7 , the computer program instructions further defining the steps of:
extracting an IP address from encrypted network request; and
updating a mapping using the IP address and the network service, the mapping comprising a mapping of IP addresses to network services.
12. The non-transitory computer readable storage medium of claim 7 , wherein the property is selected from the group consisting of response size, response time, network route, and uniform resource identifier (URI) header length.
13. A device comprising:
a processor; and
a storage medium for tangibly storing thereon program logic for execution by the processor, the stored program logic causing the processor to perform operations of:
intercepting an encrypted network request, the encrypted network request transmitted by a client device to a network endpoint to establish a secure session,
identifying a network service associated with the network endpoint based on unencrypted properties of the encrypted network request, the encrypted network request comprising an encrypted portion and a plaintext portion, the unencrypted properties determined based on the plaintext portion,
identifying an activity performed during the secure session by:
capturing a series of subsequent encrypted network requests issued by the client device to the network endpoint during a transaction burst occurring during the secure session,
generating a fingerprint based on the series of subsequent encrypted network requests, the fingerprint comprising at least one aggregated feature computed using properties of the series of subsequent encrypted network requests,
inputting the fingerprint and an identification of the network service to a predictive model, the predictive model trained using transaction bursts by extracting one or more transaction properties from the transaction burst, assigning labels to the one or more transaction properties, a given label comprising one or more of a network service and an action, and training the predictive model with labeled data, and
using an output of the predictive model as the activity, and
updating a catalog of network interactions using the network service and the activity,
extracting a property from the encrypted network request, and
updating a mapping using the property and the network service.
14. The device of claim 13 , the identifying the network service comprising extracting a host and domain from a plaintext portion of a TLS handshake.
15. The device of claim 13 , the one or more transaction properties comprising a property selected from the group consisting of: a transmission control protocol (TCP) port; an Internet Protocol (IP) address space; a size of a datagram; a response time; a number of requests in the transaction burst; and a network route trace.
16. The device of claim 13 , stored program logic further causing the processor to perform the operations of:
extracting an IP address from encrypted network request; and
updating a mapping using the IP address and the network service, the mapping comprising a mapping of IP addresses to network services.
17. The device of claim 13 , wherein the property is selected from the group consisting of response size, response time, network route, and uniform resource identifier (URI) header length; and updating a mapping using the property and the network service, the mapping comprising a mapping of properties to network services.
18. The method of claim 1 , wherein the mapping comprises a mapping of properties to network services.
19. The non-transitory computer readable storage medium of claim 7 , wherein the mapping comprises a mapping of properties to network services.
20. The device of claim 13 , wherein the mapping comprises a mapping of properties to network services.