System and method for two-tier reporting for cloud computing realms
In accordance with an embodiment, systems and methods are provided for two-tier reporting for cloud computing realms. An exemplary method can deploy a central instance of an analytics service in a central cloud realm. The method can further deploy a respective instance of the analytics service in each of a plurality of cloud realms. The method can implement a respective different data pipeline for each deployed respective instance of the analytics service, wherein a respective different data pipeline is configured to perform at least one of ingest or transform data for the deployed respective instance of the analytics service, said data being descriptive of use of services associated with a respective cloud realm of the plurality of cloud realms.
1 . A method, comprising:
deploying a central analytics service instance in a central cloud realm;
deploying a plurality of cloud realm-specific analytics service instances respectively in each of a plurality of cloud realms, each of the plurality of cloud realm-specific analytics service instances independently executing within a respective cloud realm of the plurality of cloud realms;
implementing a plurality of data pipelines respectively in the plurality of cloud realm-specific analytics service instances,
wherein a first data pipeline, of the plurality of data pipelines, is (a) configured in accordance with a first set of dimensions associated with a first cloud realm-specific analytics service instance of the plurality of cloud realm-specific analytics service instances and (b) configured to perform at least one of ingesting or transforming a first set of usage data corresponding to the first cloud realm-specific analytics service instance and associated with a first cloud realm of the plurality of cloud realms;
wherein a second data pipeline, of the plurality of data pipelines, is (a) configured in accordance with a second set of dimensions associated with a second cloud realm- specific analytics service instance of the plurality of cloud realm-specific analytics service instances and (b) configured to perform at least one of ingesting or transforming a second set of usage data corresponding to the second cloud realm-specific analytics service instance and associated with a second cloud realm of the plurality of cloud realms;
wherein the first cloud realm-specific analytics service instance is different from and operates independently from the second cloud realm-specific analytics service instance;
wherein the first data pipeline is different from and operates independently from the second data pipeline;
generating a first realm-specific report by the first cloud realm-specific analytics service instance based on the first set of usage data;
generating a second realm-specific report by the second cloud realm-specific analytics service instance based on the second set of usage data;
receiving, at the central analytics service instance via cross-realm communications using the first data pipeline, at least a subset of the first set of usage data;
receiving, at the central analytics service instance via cross-realm communications using the second data pipeline, at least a subset of the second set of usage data; and
generating a central report by the central analytics service instance based at least on the subset of first set of usage data and the subset of the second set of usage data;
wherein the method is performed on one or more computers comprising one or more microprocessors.
2 . The method of claim 1 , further comprising:
transforming the first set of usage data by the first data pipeline according to first set of dimensions associated with the first cloud realm and the first cloud realm-specific analytics service instance; and
transforming the second set of usage data by the second data pipeline according to a second set of dimensions associated with the second cloud realm and the second cloud realm-specific analytics service instance.
3 . The method of claim 1 , wherein the first cloud realm-specific analytics service instance does not have access to the second set of usage data, and wherein the second cloud realm-specific analytics service instance does not have access to the first set of usage data.
4 . The method of claim 1 , wherein the at least a subset of the first set of usage data and the at least a subset of the second set of usage data received at the central analytics service instance includes at least metering data for service usage calculation, or usage data for capacity planning, or reservation data for capacity planning.
5 . The method of claim 1 , further comprising:
prior to receiving the at least a subset of the first set of usage data, filtering the at least a subset of the first set of usage data to remove information that is sensitive with respect to the first cloud realm; and
prior to receiving the at least a subset of the second set of usage data, filtering the at least a subset of the second set of usage data to remove information that is sensitive with respect to the second cloud realm.
6 . The method of claim 1 , wherein the first cloud realm comprises one or more data centers.
7 . The method of claim 1 , wherein deploying the first analytics service instance comprises:
deploying, from the central cloud realm, a bootstrap application within the first cloud realm;
pulling, via the bootstrap application deployed within the first cloud realm, identifying data associated with the first cloud realm to a temporary storage associated with the central cloud realm; and
upon deploying the first analytics service instance, deleting the temporary storage associated with the central cloud realm.
8 . A system, comprising:
one or more computers, each comprising one or more microprocessors,
wherein the one or more computers are configured to perform a method comprising:
deploying a central analytics service instance in a central cloud realm;
deploying a plurality of cloud realm-specific analytics service instances respectively in each of a plurality of cloud realms, each of the plurality of cloud realm-specific analytics service instances independently executing within a respective cloud realm of the plurality of cloud realms;
implementing a plurality of data pipelines respectively in the plurality of cloud realm-specific analytics service instances,
wherein a first data pipeline, of the plurality of data pipelines, is (a) configured in accordance with a first set of dimensions associated with a first cloud realm- specific analytics service instance of the plurality of cloud realm-specific analytics service instances and (b) configured to perform at least one of ingesting or transforming a first set of usage data corresponding to the first cloud realm-specific analytics service instance and associated with a first cloud realm of the plurality of cloud realms;
wherein a second data pipeline, of the plurality of data pipelines, is (a) configured in accordance with a second set of dimensions associated with a second cloud realm-specific analytics service instance of the plurality of cloud realm-specific analytics service instances and (b) configured to perform at least one of ingesting or transforming a second set of usage data corresponding to the second cloud realm-specific analytics service instance and associated with a second cloud realm of the plurality of cloud realms;
wherein the first cloud realm-specific analytics service instance is different from and operates independently from the second cloud realm-specific analytics service instance;
wherein the first data pipeline is different from and operates independently from the second data pipeline;
generating a first realm-specific report by the first cloud realm- specific analytics service instance based on the first set of usage data;
generating a second realm-specific report by the second cloud realm-specific analytics service instance based on the second set of usage data;
receiving, at the central analytics service instance via cross-realm communications using the first data pipeline, at least a subset of the first set of usage data;
receiving, at the central analytics service instance via cross-realm communications using the second data pipeline, at least a subset of the second set of usage data; and
generating a central report by the central analytics service instance based at least on the subset of first set of usage data and the subset of the second set of usage data.
9 . The system of claim 8 , wherein the method further comprises:
transforming the first set of usage data by the first data pipeline according to first set of dimensions associated with the first cloud realm and the first cloud realm-specific analytics service instance; and
transforming the second set of usage data by the second data pipeline according to a second set of dimensions associated with the second cloud realm and the second cloud realm-specific analytics service instance.
10 . The system of claim 8 , wherein the first cloud realm-specific analytics service instance does not have access to the second set of usage data, and wherein the second cloud realm-specific analytics service instance does not have access to the first set of usage data.
11 . The system of claim 8 , wherein the at least a subset of the first set of usage data and the at least a subset of the second set of usage data received at the central analytics service instance includes at least metering data for service usage calculation, or usage data for capacity planning, or reservation data for capacity planning.
12 . The system of claim 8 , wherein the method further comprises:
prior to receiving the at least a subset of the first set of usage data, filtering the at least a subset of the first set of usage data to remove information that is sensitive with respect to the first cloud realm; and
prior to receiving the at least a subset of the second set of usage data, filtering the at least a subset of the second set of usage data to remove information that is sensitive with respect to the second cloud realm.
13 . The system of claim 8 , wherein the first cloud realm comprises one or more data centers.
14 . The system of claim 8 , wherein deploying the first analytics service instance comprises:
deploying, from the central cloud realm, a bootstrap application within the first cloud realm;
pulling, via the bootstrap application deployed within the first cloud realm, identifying data associated with the first cloud realm to a temporary storage associated with the central cloud realm; and
upon deploying the first analytics service instance, deleting the temporary storage associated with the central cloud realm.
15 . A non-transitory computer readable storage medium having instructions thereon, which when read and executed cause a computer comprising one or more microprocessors to perform steps comprising:
deploying a central analytics service instance in a central cloud realm;
deploying a plurality of cloud realm-specific analytics service instances respectively in each of a plurality of cloud realms, each of the plurality of cloud realm-specific analytics service instances independently executing within a respective cloud realm of the plurality of cloud realms;
implementing a plurality of data pipelines respectively in the plurality of cloud realm-specific analytics service instances,
wherein a first data pipeline, of the plurality of data pipelines, is (a) configured in accordance with a first set of dimensions associated with a first cloud realm-specific analytics service instance of the plurality of cloud realm-specific analytics service instances and (b) configured to perform at least one of ingesting or transforming a first set of usage data corresponding to the first cloud realm-specific analytics service instance of the analytics service, said data being descriptive of use of services and associated with a respective a first cloud realm of the plurality of cloud realms;
wherein a second data pipeline, of the plurality of data pipelines, is (a) configured in accordance with a second set of dimensions associated with a second cloud realm-specific analytics service instance of the plurality of cloud realm-specific analytics service instances and (b) configured to perform at least one of ingesting or transforming a second set of usage data corresponding to the second cloud realm-specific analytics service instance and associated with a second cloud realm of the plurality of cloud realms;
wherein the first cloud realm-specific analytics service instance is different from and operates independently from the second cloud realm-specific analytics service instance;
wherein the first data pipeline is different from and operates independently from the second data pipeline;
generating a first realm-specific report by the deployed respective first cloud realm-specific analytics service instance of the analytics service based on the first set of usage data;
generating a second realm-specific report by the second cloud realm-specific analytics service instance based on the second set of usage data;
receiving, at the central analytics service instance via cross-realm communications using the first data pipeline, at least a subset of the first set of usage data;
receiving, at the central analytics service instance via cross-realm communications using the second data pipeline, at least a subset of the second set of usage data; and
generating a central report by the central analytics service instance based at least on the subset of first set of usage data and the subset of the second set of usage data.
16 . The non-transitory computer readable storage medium of claim 15 , the steps further comprising:
transforming the first set of usage data by the first data pipeline according to first set of dimensions associated with the first cloud realm and the first cloud realm-specific analytics service instance; and
transforming the second set of usage data by the second data pipeline according to a second set of dimensions associated with the second cloud realm and the second cloud realm-specific analytics service instance.
17 . The non-transitory computer readable storage medium of claim 15 , wherein the first cloud realm-specific analytics service instance does not have access to the second set of usage data, and wherein the second cloud realm-specific analytics service instance does not have access to the first set of usage data.
18 . The non-transitory computer readable storage medium of claim 15 , wherein the at least a subset of the first set of usage data and the at least a subset of the second set of usage data received at the central analytics service instance includes at least metering data for service usage calculation, or usage data for capacity planning, or reservation data for capacity planning.
19 . The non-transitory computer readable storage medium of claim 15 , the steps further comprising:
prior to receiving the at least a subset of the first set of usage data, filtering the at least a subset of the first set of usage data to remove information that is sensitive with respect to the first cloud realm; and
prior to receiving the at least a subset of the second set of usage data, filtering the at least a subset of the second set of usage data to remove information that is sensitive with respect to the second cloud realm.
20 . The non-transitory computer readable storage medium of claim 15 , wherein the first cloud realm comprises one or more data centers.
21 . The non-transitory computer readable storage medium of claim 15 , wherein deploying the first analytics service instance comprises:
deploying, from the central cloud realm, a bootstrap application within the first cloud realm;
pulling, via the bootstrap application deployed within the first cloud realm, identifying data associated with the first cloud realm to a temporary storage associated with the central cloud realm; and
upon deploying the first analytics service instance, deleting the temporary storage associated with the central cloud realm.