System and method for generating data sets for learning to identify user actions
Embodiments for generating appropriate data sets for learning to identify user actions. A user uses one or more applications over a suitable period of time. As the user uses the applications, a monitoring device, acting as a “man-in-the-middle,” intermediates the exchange of encrypted communication between the applications and the servers that serve the applications. The monitoring device obtains, for each action performed by the user, two corresponding (bidirectional) flows of communication: an encrypted flow, and an unencrypted flow. Since the unencrypted flow indicates the type of action that was performed by the user, the correspondence between the encrypted flow and the unencrypted flow may be used to automatically label the encrypted flow, without decrypting the encrypted flow. Features of the encrypted communication may then be stored in association with the label to automatically generate appropriately-sized learning set for each application of interest.
1. A system, for use with a monitoring device intermediating an exchange, between an application and a server, of encrypted communication that was generated in response to one or more actions performed by a user of the application, the system comprising:
a network interface; and
at least one processor, configured:
to receive from the monitoring device, via the network interface, the encrypted communication, and corresponding unencrypted communication that was derived, by the monitoring device, from the encrypted communication;
to extract a type of user action from the unencrypted communication;
to ascertain a correspondence between the unencrypted communication and the encrypted communication, and
to infer that encrypted communication is associated with the extracted type based on the correspondence between the unencrypted communication and the encrypted communication.
2. The system according to claim 1 , wherein the at least one processor configured to extract a type of user action from the unencrypted communication comprises the at least one processor configured to search the unencrypted communication for fieldnames that indicate the type of user action.
3. The system according to claim 1 , wherein the correspondence between the unencrypted communication and the encrypted communication is ascertained without decrypting the encrypted communication.
4. The system according to claim 1 , wherein the at least one processor is further configured to apply labels, which indicate the type of user action, to the encrypted communication, by storing features of the encrypted communication in association with the labels.
5. The system according to claim 1 ,
wherein the encrypted communication includes a plurality of encrypted blocks, each of the encrypted blocks including at least a first group of packets sent from the application to the server, and a second group of packets sent from the server to the application in response to the first group of packets,
wherein the unencrypted communication includes a plurality of unencrypted blocks, each of the unencrypted blocks including at least a first group of packets sent from the application to the server, and a second group of packets sent from the server to the application in response to the first group of packets.
6. The system according to claim 5 , wherein the processor is configured to apply labels which indicate the type of user action to the encrypted communication by storing features of each encrypted block of the encrypted blocks in association with a respective one of the labels that indicates the type of the action in response to which the encrypted block was generated.
7. The system according to claim 5 , wherein the processor is further configured to separate the encrypted communication into the encrypted blocks, based on respective durations of time between successive pairs of the encrypted blocks being greater than a threshold.
8. The system according to claim 5 , wherein the processor is configured to ascertain that any particular one of the encrypted blocks corresponds to a particular one of the unencrypted blocks, based on a duration of time between the particular one of the encrypted blocks and the particular one of the unencrypted blocks being less than a threshold.
9. The system according to claim 5 , wherein the processor is configured to ascertain the correspondence between the encrypted blocks and the unencrypted blocks by comparing respective durations of the encrypted blocks to respective durations of the unencrypted blocks.
10. The system according to claim 5 , wherein the processor is configured to ascertain the correspondence between the encrypted blocks and the unencrypted blocks by comparing respective amounts of data in the encrypted blocks to respective amounts of data in the unencrypted blocks.
11. A method for use with a monitoring device intermediating an exchange, between an application and a server, of encrypted communication that was generated in response to one or more actions performed by a user of the application, the method comprising:
receiving from the monitoring device the encrypted communication, and corresponding unencrypted communication that was derived from the encrypted communication;
extracting a type of user action from the unencrypted communication;
ascertaining a correspondence between the unencrypted communication and the encrypted communication, and
inferring that encrypted communication is associated with the extracted type based on the correspondence between the unencrypted communication and the encrypted communication.
12. The method according to claim 11 , wherein extracting a type of user action from the unencrypted communication comprises searching the unencrypted communication for fieldnames that indicate the type of user action.
13. The method according to claim 11 , wherein the correspondence between the unencrypted communication and the encrypted communication is ascertained without decrypting the encrypted communication.
14. The method according to claim 11 , wherein the at least one processor is further configured to apply labels, which indicate the type of user action, to the encrypted communication, by storing features of the encrypted communication in association with the labels.
15. The method according to claim 11 ,
wherein the encrypted communication includes a plurality of encrypted blocks, each of the encrypted blocks including at least a first group of packets sent from the application to the server, and a second group of packets sent from the server to the application in response to the first group of packets,
wherein the unencrypted communication includes a plurality of unencrypted blocks, each of the unencrypted blocks including at least a first group of packets sent from the application to the server, and a second group of packets sent from the server to the application in response to the first group of packets.
16. The method according to claim 15 , further comprising applying labels which indicate the type of user action to the encrypted communication by storing features of each encrypted block of the encrypted blocks in association with a respective one of the labels that indicates the type of the action in response to which the encrypted block was generated.
17. The method according to claim 15 , further comprising separating the encrypted communication into the encrypted blocks, based on respective durations of time between successive pairs of the encrypted blocks being greater than a threshold.
18. The method according to claim 15 , further comprising ascertaining that any particular one of the encrypted blocks corresponds to a particular one of the unencrypted blocks, based on a duration of time between the particular one of the encrypted blocks and the particular one of the unencrypted blocks being less than a threshold.
19. The method according to claim 15 , further comprising ascertaining the correspondence between the encrypted blocks and the unencrypted blocks by comparing respective durations of the encrypted blocks to respective durations of the unencrypted blocks.
20. The method according to claim 15 , further comprising ascertaining the correspondence between the encrypted blocks and the unencrypted blocks by comparing respective amounts of data in the encrypted blocks to respective amounts of data in the unencrypted blocks.