Data ingestion into top-level shards
Systems, methods, and products for data ingestion into top-level shards that include a self-contained infrastructure for detecting anomalies in a cloud-computing environment, including an intake database for receiving input data from the cloud-computing environment, an anomaly detection module, an output database storing results of the processing of the anomaly detection module, and an alert generation module for generating event alerts based on the anomaly detection, including: associating a domain-independent top-level shard with a customer identifier; receiving data corresponding to the customer identifier; and storing the received data in the top-level shard.
1 . A computer program product for data ingestion into top-level shards, the computer program product disposed on a computer readable medium, the computer program product including computer program instructions that, when executed by one or more processors, carry out the steps of:
receiving one or more configuration inputs via a graphical user interface, the configuration inputs including shard configuration details specifying characteristics and properties of a shard;
selecting a domain-independent top-level shard based on the shard configuration details, wherein the top-level shard comprises a self-contained infrastructure for ingesting and processing data to detect anomalies in a plurality of independent cloud-computing environment domains;
receiving data corresponding to a customer identifier associated with the domain-independent top-level shard; and
storing the received data in the top-level shard included within a first of the plurality cloud-computing environment domains.
2 . The computer program product of claim 1 further comprising computer program instructions that, when executed, carry out the step of:
generating a domain-independent top-level shard in a cloud-based storage system.
3 . The computer program product of claim 1 wherein the top-level shard is included in a first domain, and wherein storing the received data in the top-level shard further comprises:
migrating the top-level shard from the first domain to a second domain, wherein the first domain and the second domain correspond to different failure domains.
4 . The computer program product of claim 3 , wherein migrating the top-level shard further comprises:
generating a second domain-independent top-level shard in the second domain;
copying customer data from the top-level shard to the second top-level shard; and
redirecting data input and output traffic to the second top-level shard instead of the top-level shard.
5 . The computer program product of claim 1 , wherein storing the received data in the top-level shard further comprises:
storing a first portion of the data corresponding to the customer identifier in a first top-level shard; and
storing a second portion of the data corresponding to the customer identifier in a second top-level shard.
6 . The computer program product of claim 1 further comprising storing data corresponding to a second customer identifier in the top-level shard.
7 . The computer program product of claim 6 , wherein storing data corresponding to the second customer identifier in the top-level shard further comprises:
determining a value for an attribute of the received data corresponding to the customer identifier; and
selecting the top-level shard from a plurality of top-level shards based on the determined value for the attribute.
8 . The computer program product of claim 1 , wherein the configuration inputs further comprise the customer identifier.
9 . The computer program product of claim 1 , wherein receiving data corresponding to the customer identifier further comprises:
receiving the data from an agent executing as a service on a computing device associated with the customer.
10 . The computer program product of claim 1 , wherein the domain-independent top-level shard comprises a self-contained infrastructure including an intake database for receiving input data from the cloud-computing environment, an anomaly detection module, an output database storing results of the processing of the anomaly detection module, and an alert generation module for generating event alerts based on the anomaly detection.
11 . A method of data ingestion into top-level shards, the method comprising:
receiving one or more configuration inputs via a graphical user interface, the configuration inputs including shard configuration details specifying characteristics and properties of a shard;
selecting a domain-independent top-level shard based on the shard configuration details, wherein the top-level shard comprises a self-contained infrastructure for ingesting and processing data to detect anomalies in a plurality of independent cloud-computing environment domains;
receiving data corresponding to a customer identifier associated with the domain-independent top-level shard; and
storing the received data in the top-level shard included within a first of the plurality cloud-computing environment domains.
12 . The method of claim 11 , further comprising:
generating a domain-independent top-level shard in a cloud-based storage system.
13 . The method of claim 11 , wherein the top-level shard is included in a first domain, and wherein storing the received data in the top-level shard further comprises:
migrating the top-level shard from the first domain to a second domain, wherein the first domain and the second domain correspond to different failure domains.
14 . The method of claim 13 , wherein migrating the top-level shard further comprises:
generating a second domain-independent top-level shard in the second domain;
copying customer data from the top-level shard to the second top-level shard; and
redirecting data input and output traffic to the second top-level shard instead of the top-level shard.
15 . The method of claim 11 , wherein storing the received data in the top-level shard further comprises:
storing a first portion of the data corresponding to the customer identifier in a first top-level shard and storing a second portion of the data corresponding to the customer identifier in a second top-level shard.
16 . A system comprising one or more processors to execute computer program instructions to carry out the steps of:
receiving one or more configuration inputs via a graphical user interface, the configuration inputs including shard configuration details specifying characteristics and properties of a shard;
selecting a domain-independent top-level shard based on the shard configuration details, wherein the top-level shard comprises a self-contained infrastructure for ingesting and processing data to detect anomalies in a plurality of independent cloud-computing environment domains;
receiving data corresponding to a customer identifier associated with the domain-independent top-level shard; and
storing the received data in the top-level shard included within a first of the plurality cloud-computing environment domains.
17 . The system of claim 16 further comprising computer program instructions that, when executed, carry out the step of:
generating a domain-independent top-level shard in a cloud-based storage system.
18 . The system of claim 16 wherein the top-level shard is included in a first domain, and wherein storing the received data in the top-level shard further comprises:
migrating the top-level shard from the first domain to a second domain, wherein the first domain and the second domain correspond to different failure domains.
19 . The system of claim 18 , wherein migrating the top-level shard further comprises:
generating a second domain-independent top-level shard in the second domain;
copying customer data from the top-level shard to the second top-level shard; and
redirecting data input and output traffic from one or more customer computing devices to the second top-level shard instead of the top-level shard.
20 . The system of claim 16 , wherein storing the received data in the top-level shard further comprises:
storing a first portion of the data corresponding to the customer identifier in a first top-level shard and storing a second portion of the data corresponding to the customer identifier in a second top-level shard.