Group-based event analysis service
A group-based event analysis service may be implemented by a provider network to monitor service-to-service event streams to identify behavior that is abnormal for a particular group of event streams. Event streams may be classified into groups based on the computing resources or services that generate the events, the behavior patterns of the events in the event stream, and other information about the event streams. Deviations in behavior of individual members of a group relative to group behavior may be used to identify anomalous and/or potentially harmful behavior.
1 . A system, comprising:
a service provider network, comprising:
a first set of computing resources configured to implement, at least in part, a first service of the service provider network; and
one or more other sets of computing resources configured to implement, at least in part, one or more other services of the service provider network,
wherein the first set of computing resources and the one or more other sets of computing resources communicate with one another via one or more service-to-service event streams; and
wherein at least one computing resource of the service provider network is configured to implement a group-based event analysis service configured to:
obtain data related to the one or more service-to-service event streams;
designate groupings of events or event flows within the one or more service-to-service event streams;
determine whether an event or an event flow of a given one of the designated groupings comprises anomalous activity relative to other events or event flows included in the given designated grouping; and
in response to determining an event or an event flow of a given designated grouping comprises anomalous activity, cause one or more processes originating the event or event flow to be terminated.
2 . The system of claim 1 , wherein the least one computing resource of the service provider network configured to implement the group-based event analysis service is further configured to:
determine whether the anomalous activity is considered harmful,
wherein said termination of the one or more processes originating the anomalous activity is further performed in response to a determination that the anomalous activity is considered harmful activity.
3 . The system of claim 1 , wherein the service provider network is configured to:
automatically perform one or more failover operations in response to the termination of the one or more processes, wherein the one or more failover operations comprise:
instantiating the one or more processes on a replacement computing resource that is a different computing resource than a computing resource for which the one or more processes were terminated.
4 . The system of claim 1 , wherein the group-based event analysis service is further configured to:
store a snapshot of a state of a computing resource executing the terminated one or more processes.
5 . The system of claim 1 , wherein to designate groupings of the event or event flows of the one or more service-to-service event streams based on the received data, the group-based event analysis service implements a k-means grouping structure or a fuzzy-c-means grouping structure.
6 . The system of claim 1 , wherein the first set of computing resources and at least one of the one or more other sets of computing resources use a same code base.
7 . The system of claim 1 , wherein the first set of computing resources of the first service and the one or more sets of computing resources of the one or more other services communicate with customers via service-to-customer event streams, and wherein the service-to-service event streams are distinct streams from the service-to-customer streams.
8 . The system of claim 1 , wherein the group-based event analysis service is further configured to:
generate multi-dimensional vector representations for the events or event flows based on the received data, wherein:
the received data comprises information for determining a plurality of attributes corresponding to the events or event-flows of the respective one or more service-to-service event streams; and
dimensions, of a multi-dimensional vector space in which the multi-dimensional vector representations are generated, correspond to respective ones of the attributes.
9 . A method, comprising:
obtaining data related to one or more service-to-service event streams;
designating, based on the received data, groupings of events or event flows within the one or more service-to-service event streams;
determining a particular service-to-service event stream comprises anomalous activity relative to a particular grouping that includes the particular service-to-service event stream based on the received data; and
causing one more mitigation actions to be performed in response to determining a presence of the anomalous activity in the particular service-to-service event stream.
10 . The method of claim 9 , wherein the one or more mitigation actions comprise:
terminating one or more processes originating the anomalous activity.
11 . The method of claim 9 , wherein said initiating the one or more mitigation actions is further based on a determination that the anomalous activity is activity that is considered harmful.
12 . The method of claim 11 , wherein the one or more mitigation actions further comprise issuing an alert message indicating the presence of activity that is considered harmful.
13 . The method of claim 9 , wherein a probabilistic grouping technique is used to designate groupings of the events or event flows of the one or more service-to-service event streams based on the received data.
14 . The method of claim 9 , further comprising:
evaluating the groupings for redesignation in response to determining the particular service-to-service event stream comprises anomalous activity;
redesignating the groupings of the events or event flows of the one or more service-to-service event streams based on the evaluation of the groupings.
15 . The method of claim 9 , wherein the service-to-service event streams comprise communications between:
a first set of computing resources of a first service of a service provider network;
a second set of computing resources of a second service of the service provider; and
wherein the first set of computing resources and the second set of computing resources share a common code base.
16 . One or more non-transitory computer-readable media storing program instructions that, when executed on or across one or more processors, cause the one or more processors to:
obtain data related to one or more service-to-service event streams;
designate, based on the received data, groupings of events or event flows of the one or more service-to-service event streams; and
determine a particular service-to-service event stream comprises anomalous activity relative to a particular grouping that includes the particular service-to-service event stream based on the received data.
17 . The one or more non-transitory computer-readable media of claim 16 , wherein the program instructions, when executed on or across the one or more processors, further cause the one or more processors to:
initiate one or more mitigation actions based on the determination that the particular service-to-service event stream comprises anomalous activity, wherein the one or more mitigation actions comprise terminating one or more processes originating events included in the particular service-to-service event stream.
18 . The one or more non-transitory computer-readable media of claim 16 , wherein to designate the groupings, the program instructions, when executed on or across the one or more processors, cause the one or more processors to:
determine attributes for the one or more service-to-service event streams; and
designate the groupings based on statistical similarities between the attributes for the one or more service-to-service event streams.
19 . The one or more non-transitory computer-readable media of claim 16 , wherein the program instructions, when executed on or across the one or more processors, further cause the one or more processors to:
generate a multi-dimensional vector, wherein:
the received data related to the plurality of service-to-service event streams comprises a plurality of attributes of the particular service-to-service event stream; and
an attribute of the plurality of attributes corresponds to a dimension of the multi-dimensional vector.
20 . The one or more non-transitory computer-readable media of claim 16 , wherein the program instructions, when executed on or across the one or more processors, further cause the one or more processors to:
redesignate groupings of the events or event flows of the one or more service-to-service event streams based on additionally received data,
wherein the program instructions determine, based on the additionally received data, that a given service-to-service event stream comprises anomalous activity relative to a given grouping that includes the given service-to-service event stream.