Software container registry inspection
A request to a scan a software image for specified criteria is received, the software image comprising layers stored in a first data store. Metadata in a second data store, different from the first data store, is searched through to obtain information corresponding to the software image. A first set of the layers that matches the specified criteria is determined, based at least in part on the information. The first set of layers is marked as un-referenceable. Asynchronous to fulfillment of the request, a second set of layers of the layers to be deleted is determined, based at least in part on the metadata, the second set of layers including layers marked as un-referenceable, and the second set of layers is deleted.
1. A system, comprising:
one or more processors; and
memory including executable instructions that, as a result of execution by the one or more processors, cause the system to:
receive a request to perform a scan of a set of container images stored in at least one repository, a container image of the set of container images comprising encrypted image layers assigned to an account, the request including criteria for identifying insecure image layers associated with a security vulnerability; and
in response to receiving the request:
search a set of manifests stored in a database of a structured data store to obtain content-addressable identifiers for the encrypted image layers, the set of manifests comprising metadata about the set of container images;
determine, based at least in part on the content-addressable identifiers, an insecure image layer at least in part by causing the system to:
decrypt the encrypted image layers using a decryption key to form decrypted image layers, the decryption key being obtained via an entity associated with the account; and
determine the insecure image layer from one of the decrypted image layer that is associated with a match to the criteria; and
flag the insecure image layer as un-referenceable; and
as a result of an occurrence of a current time corresponding to a time scheduled for performing a deletion operation:
determine, by analyzing the set of manifests, a superset of image layers that includes one or more image layers that are flagged as un-referenceable; and
delete the superset of image layers.
2. The system of claim 1 , wherein:
the criteria comprise a specified content-addressable identifier;
the request is a request to search the content-addressable identifiers for an occurrence of the specified content-addressable identifier; and
the executable instructions that cause the system to determine, based at least in part on the content-addressable identifiers, the insecure image layer further cause the system to determine, based at least in part on the content-addressable identifiers, the insecure image layer from a layer that has a content-addressable identifier that matches the specified content-addressable identifier.
3. The system of claim 1 , wherein:
the criteria comprise a digital fingerprint;
the request is a request to search one or more files in the encrypted image layers for an occurrence of the digital fingerprint; and
the executable instructions that cause the system to determine, based at least in part on the content-addressable identifiers, the insecure image layer further cause the system to determine the insecure image layer by identifying, as the insecure image layer, a layer of the decrypted image layers in which at least one file has the occurrence of the digital fingerprint.
4. The system of claim 1 , wherein flagging the insecure image layers as un-referenceable:
causes the insecure image layer flagged as un-referenceable to be inaccessible to customers of a computing resource service provider that hosts the system; and
prevents the system from launching a container image that includes the insecure image layer flagged as un-referenceable.
5. The system of claim 1 , wherein the criteria includes a digital fingerprint associated with malicious software (malware) or sensitive data.
6. The system of claim 1 , wherein the executable instructions further cause the system to, as a result of the performing the scan of the set of container images, update a table to include a timestamp that indicates a time at which the set of container images were scanned.
7. A computer-implemented method, comprising:
receiving a request to scan a software image for a match to specified criteria, the software image comprising image layers stored as encrypted image layers in a data object store assigned to an account;
searching metadata in a structured data store, different from the data object store, to obtain a set of identifiers for the image layers;
determining, based at least in part on the set of identifiers, that a first set of the image layers is associated with a match to the specified criteria at least in part by:
obtaining a decryption key from an entity associated with the account;
decrypting the encrypted image layers using the decryption key to form decrypted image layers; and
determining the first set of the image layers from one or more layers of the decrypted image layers that are associated with the match to the specified criteria;
marking the first set of the image layers as un-referenceable;
detecting an occurrence of an event that triggers deletion of un-referenceable image layers;
determining, by analyzing the metadata, a set of un-referenceable layers of the image layers, the set of un-referenceable layers including the first set of the image layers and a second set of image layers comprising image layers stored in the data object store that are associated with an untagged software image; and
deleting the set of un-referenceable layers.
8. The computer-implemented method of claim 7 , wherein:
the data object store assigned to the account is maintained by a computing resource service provider that provides the software image to one or more customers of the computing resource service provider; and
as a result of determining that the first set of the image layers is associated with the match to the specified criteria, the method further comprises:
determining a set of the one or more customers having instances in which the software image has been launched; and
notifying the set of the one or more customers of a potential vulnerability found with the software image.
9. The computer-implemented method of claim 7 , wherein determining the set of un-referenceable layers includes:
searching the metadata of the structured data store for one or more layers of the image layers that are unassociated with a tagged software image; and
identifying, for deletion, the one or more layers as the second set of image layers.
10. The computer-implemented method of claim 7 , wherein the event is one of an instruction from an entity associated with the account received through an application programming interface or a clock event wherein a current time corresponds to a predetermined schedule for performing a deletion operation.
11. The computer-implemented method of claim 7 , wherein the metadata includes a manifest for the software image that includes, for each image layer of the image layers, a content-addressable identifier that uniquely corresponds to the image layer and a checksum for verifying integrity of the image layer.
12. The computer-implemented method of claim 7 , wherein:
the data object store includes a set of repositories associated with the account;
the method further comprises:
receiving, through an application programming interface, an indication from a customer of a computing resource service provider associated with the account, a selection of one or more repositories of the set of repositories for the scan; and
determining that the first set of the image layers is associated with the match to the specified criteria further includes determining the first set of the image layers from image layers of the image layers that are stored in the one or more repositories.
13. The computer-implemented method of claim 7 , wherein:
the method further comprises receiving, through an application programming interface, an indication from a customer of one or more levels of vulnerabilities for the scan; and
determining that the first set of the image layers is associated with the match to the specified criteria further includes determining that the match to the specified criteria is associated with the one or more levels of vulnerabilities indicated by the customer.
14. A one or more non-transitory computer-readable storage media having collectively stored thereon executable instructions that, as a result of execution by one or more processors of a distributed computer system, cause the distributed computer system to at least:
receive a request to scan a software image for specified criteria, the software image comprising image layers stored in a first data store as an encrypted software image;
search through metadata in a second data store, different from the first data store, to obtain information corresponding to the software image;
determine, based at least in part on the information, a first set of the image layers that matches the specified criteria at least in part by causing the distributed computer system to:
decrypt the encrypted software image using a cryptographic key shared between an entity and the distributed computer system to form decrypted layers, the entity being associated with the encrypted software image through an account that is hosted by a computing resource service provider that hosts the distributed computer system; and
determine the first set of the image layers at least in part from layers of the decrypted layers that contain one or more files that match reference criteria;
mark the first set of the image layers as un-referenceable; and
asynchronous to fulfilment of the request:
determine, based at least in part on the metadata, a second set of the image layers to be deleted, the second set of the image layers including the first set of the image layers; and
delete the second set of the image layers.
15. The one or more non-transitory computer-readable storage media of claim 14 , wherein the executable instructions further include executable instructions that cause the distributed computer system to:
determine, based at least in part on the metadata, a third set of layers from one or more layers of the image layers that are unlinked to an image that has a tag; and
delete the third set of layers.
16. The one or more non-transitory computer-readable storage media of claim 14 , wherein an event that triggers the distributed computer system to determine the second set of the image layers to be deleted is one of:
receiving, from a device associated with a customer of a computing resource service provider hosting the distributed computer system, an application programming interface request to clean a repository of the customer, the repository located in the second data store,
receiving, from the device associated with the customer, an application programming interface request to delete a particular version of the software image from the repository, or
an occurrence of a current time that corresponds to a predefined schedule for performing garbage collection.
17. The one or more non-transitory computer-readable storage media of claim 14 , wherein the executable instructions further include executable instructions that cause the distributed computer system to, in response to receiving a second request to launch the software image to run as a software container in an instance:
make a determination whether the image layers includes a layer that has been marked as un-referenceable; and
based at least in part on the determination, deny the second request.
18. The one or more non-transitory computer-readable storage media of claim 14 , wherein:
the first data store is a data object store that stores the image layers as a set of data objects; and
the second data store is structured data storage that hosts a scalable, distributed database for storing metadata about images.
19. The one or more non-transitory computer-readable storage media of claim 18 , wherein:
the specified criteria include at least one specified content-addressable identifier of a layer; and
the executable instructions that cause the distributed computer system to determine the first set of the image layers include executable instructions that cause the distributed computer system to:
search the metadata of the second data store for one or more layers of set of the image layers that have content-addressable identifiers that match the at least one specified content-addressable identifier; and
identify the one or more layers as the first set of the image layers.
20. The one or more non-transitory computer-readable storage media of claim 14 , wherein:
the specified criteria include the reference criteria for identifying one or more files in a layer; and
the executable instructions that cause the distributed computer system to determine the first set of the image layers include executable instructions that cause the distributed computer system to:
obtain the software image from the first data store;
open the image layers of the software image to form opened layers; and
determine the first set of the image layers at least in part from layers of the opened layers that contain one or more files that match the reference criteria.