Discovery of cloud-based infrastructure and resources
A computing system includes a persistent storage configured to store, as one or more configuration items and on behalf of a managed network, a table of resources provided by a remote computing system to the managed network. The computing system includes a discovery application configured to perform operations. The operations include querying a resource manager application programming interface (API) to identify resources within the remote computing system. The remote computing system is configured to generate a resource inventory file and store the resource inventory file within a storage bucket in response to the query. The resource inventory file contains a list of resources and attributes. The operations also include polling the storage bucket to determine when the resource inventory file has been fully written. In addition, the operations include parsing the resource inventory file to generate representations of the resources. Further, the operations include storing the representations in the table.
1. A computing system comprising:
a persistent storage configured to store, as one or more configuration items and on behalf of a managed network, a table of resources provided by a remote computing system to the managed network, wherein the resources are accessible by a service identifier associated with the remote computing system; and
a discovery application configured to perform operations comprising:
querying a resource manager application programming interface (API) associated with the remote computing system to identify resources within the remote computing system, wherein the remote computing system is configured to generate a resource inventory file and store the resource inventory file within a storage bucket in response to the resource manager API query, and wherein the resource inventory file contains a list of resources within the remote computing system that are accessible by the service identifier and attributes associated with each of the resources;
polling, by querying a storage bucket API associated with the remote computing system, the storage bucket to determine when the resource inventory file has been fully written to the storage bucket;
retrieving the resource inventory file from the storage bucket;
parsing the resource inventory file to generate representations of the resources; and
storing, in the persistent storage, the representations of the resources in the table.
2. The computing system of claim 1 , wherein each of the resources in the list of resources within the resource inventory file is line-delimited, and wherein parsing the resource inventory file comprises identifying one resource per line of the resource inventory file.
3. The computing system of claim 2 , wherein each of the resources is a separate object, and wherein parsing the resource inventory file comprises identifying one resource per object within the resource inventory file.
4. The computing system of claim 1 , wherein polling the storage bucket comprises transmitting repeated storage bucket API queries until a response is received that indicates the resource inventory file has been fully written to the storage bucket, and wherein the storage bucket API queries are spaced in time by a predetermined time interval.
5. The computing system of claim 4 , wherein the predetermined time interval is user-defined.
6. The computing system of claim 1 , wherein querying the resource manager API comprises providing a whitelist that indicates which types of resources within the remote computing system are to be included in the resource inventory file, and wherein the resource inventory file only contains resources having resource types listed in the whitelist.
7. The computing system of claim 6 , wherein the whitelist is user-defined.
8. The computing system of claim 1 , wherein generating the resource inventory file comprises identifying, for each respective resource within the table, a logical datacenter associated with the respective resource or a region associated with the respective resource.
9. The computing system of claim 1 , wherein the operations comprise:
requesting and receiving, from an authorized user associated with the service identifier, credentials that provide create access and edit access for storage buckets associated with the remote computing system; and
providing, to the remote computing system, the credentials.
10. The computing system of claim 1 , wherein the operations comprise:
transmitting, to the resource manager API after retrieving the resource inventory file from the storage bucket, a request to delete the resource inventory file.
11. The computing system of claim 1 , wherein the attributes include a resource name, a resource type, a unique identification number, a related region or logical datacenter, and a resource tag.
12. The computing system of claim 11 , wherein the operations comprise:
identifying a relationship between at least one of the resources and the resource tag associated with the at least one resource based on the resource type of the at least one resource; and
storing, in the table in the persistent storage, the relationship.
13. The computing system of claim 1 , wherein at least one of the resources is not associated with a logical datacenter or a related region.
14. A computer-implemented method comprising:
querying, by a discovery application, a resource manager application programming interface (API) associated with a remote computing system to identify resources within the remote computing system, wherein the remote computing system is configured to generate a resource inventory file and store the resource inventory file within a storage bucket in response to the resource manager API query, and wherein the resource inventory file contains a list of resources within the remote computing system that are accessible by a service identifier associated with the remote computing system and attributes associated with each of the resources;
polling, by the discovery application, the storage bucket to determine when the resource inventory file has been fully written to the storage bucket, wherein the polling comprises querying a storage bucket API associated with the remote computing system;
retrieving, by the discovery application, the resource inventory file from the storage bucket;
parsing, by the discovery application, the resource inventory file to generate representations of the resources; and
storing, by the discovery application, in a persistent storage as one or more configuration items and on behalf of a managed network, the representations of the resources in a table of resources.
15. The computer-implemented method of claim 14 , wherein each of the resources in the list of resources within the resource inventory file is line-delimited, and wherein parsing the resource inventory file comprises identifying one resource per line of the resource inventory file.
16. The computer-implemented method of claim 15 , wherein each of the resources is a separate object, and wherein parsing the resource inventory file comprises identifying one resource per object within the resource inventory file.
17. The computer-implemented method of claim 14 , wherein polling the storage bucket comprises transmitting repeated storage bucket API queries until a response is received that indicates the resource inventory file has been fully written to the storage bucket, and wherein the storage bucket API queries are spaced in time by a predetermined time interval.
18. The computer-implemented method of claim 14 , wherein querying the resource manager API comprises providing a whitelist that indicates which types of resources within the remote computing system are to be included in the resource inventory file, and wherein the resource inventory file only contains resources having resource types listed in the whitelist.
19. The computer-implemented method of claim 14 , comprising:
transmitting, to the resource manager API after retrieving the resource inventory file from the storage bucket, a request to delete the resource inventory file.
20. An article of manufacture including a non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by a computing system, cause the computing system to perform operations comprising:
querying, by a discovery application, a resource manager application programming interface (API) associated with a remote computing system to identify resources within the remote computing system, wherein the remote computing system is configured to generate a resource inventory file and store the resource inventory file within a storage bucket in response to the resource manager API query, and wherein the resource inventory file contains a list of resources within the remote computing system that are accessible by a service identifier associated with the remote computing system and attributes associated with each of the resources;
polling, by the discovery application, the storage bucket to determine when the resource inventory file has been fully written to the storage bucket, wherein the polling comprises querying a storage bucket API associated with the remote computing system;
retrieving, by the discovery application, the resource inventory file from the storage bucket;
parsing, by the discovery application, the resource inventory file to generate representations of the resources; and
storing, by the discovery application, in a persistent storage as one or more configuration items and on behalf of a managed network, the representations of the resources in a table of resources.