Source archive optimizations for reducing container image sizes

1 . At least one processor comprising:

one or more circuits to:

classify a source file that is to be used to generate a container image;

based at least on the source file being classified as a local source file, determine whether the source file has a size greater than a threshold size, the threshold size based at least on one or more characteristics associated with a secondary classification of the source file;

based at least on a determination that the size of the source file is greater than the threshold size, configure a local HTTP server to provide access to the source file at a network address;

generate one or more command lines that are executable to cause the source file to be retrieved from the local HTTP server using the network address; and

send the one or more command lines to a container builder to cause the container builder to generate the container image using the source file retrieved from the network address.

2 . The at least one processor of claim 1 , further comprising one or more circuits to:

determine, for another source file, that a respective size of the another source file is less than the threshold size;

copy the another source file to an intermediate work folder based at least on the respective size of the another source file being less than the threshold size;

generate another command line to copy the another source file from the intermediate work folder; and

send the another command line and the another source file from the intermediate work folder to the container builder such that the container builder further uses the another command line and the another source file to generate the container image.

3 . The at least one processor of claim 1 , further comprising one or more circuits to classify a plurality of source files into respective source types, the source types including at least one of a local source archive type, a remote source archive type, a Git source archive type, or a custom source archive type.

4 . The at least one processor of claim 1 , wherein the one or more circuits are further to, based at least on the size of the source file being greater than the threshold size, exclude the source file from a set of source files included in staging storage allocated to the container builder, the set of source files accessed by the container builder from the staging storage to build the container image.

5 . The at least one processor of claim 1 , further comprising processing circuitry to:

determine permissions data corresponding to the source file; and

store the permissions data;

wherein the one or more command lines are generated using an HTTP template configured to cause the permissions data to be retrieved for inclusion in the one or more command lines.

6 . The at least one processor of claim 5 , further comprising processing circuitry to:

compute a checksum value corresponding to the source file during a first build of the container image;

compare at least one of a current size or a current update time corresponding to the source file to a stored size or a stored update time corresponding to the source file; and

perform one of:

updating the checksum value corresponding to the source file for a second build based at least on at least one of the current size being greater than the stored size or the current update time being different from the stored update time; or

maintaining the checksum value corresponding to the source file for the second build based at least on at least one of the current size being equal to the stored size or the current update time being equal to the stored update time,

wherein the HTTP template is configured to include the checksum value in the one or more command lines.

7 . The at least one processor of claim 6 , wherein the checksum value, the stored size, and the stored update time are included in a metadata file corresponding to the source file.

8 . The at least one processor of claim 6 , further comprising processing circuitry to, during the second build, use at least one cache layer generated during the first build and corresponding to the source file when the checksum value is maintained.

9 . The at least one processor of claim 6 , the one or more processors further to:

receive a configuration file that includes user identified source archives for inclusion in a container build operation that corresponds to the container image, the user identified source archives including the source file,

wherein the classification of the source file is based at least on the receipt of the configuration file.

10 . The at least one processor of claim 1 , wherein the source file corresponds to a folder including a plurality of files, the plurality of files are labeled as a group, and the one or more command lines include a single command line for the folder such that each of the plurality of files are included in the container image using the single command line.

11 . The at least one processor of claim 10 , wherein the single command line corresponds to a single layer in the container image.

12 . The at least one processor of claim 1 , wherein the processor is comprised in at least one of:

a system for performing simulation operations;

a system for performing deep learning operations;

a system implemented using an edge device;

a system incorporating one or more virtual machines (VMs);

a system implemented at least partially in a data center; or

a system implemented at least partially using cloud computing resources.

13 . A system comprising:

one or more processing units; and

one or more memory units storing instructions that, when executed by the one or more processing units, cause the one or more processing units to execute operations comprising:

classifying a source archive file based at least on one or more attributes associated with the source archive file;

based at least on the source archive file being classified as a local file, determining, based at least on comparing the source archive file to a threshold size, that the source archive file has a size greater than the threshold size;

configuring, based at least on the size being greater than the threshold size, a local HTTP server to provide access to the source archive file at a network address;

generating one or more command lines that are executable to cause the source archive file to be retrieved from the local HTTP server using the network address; and

sending the one or more command lines to a container builder to cause the container builder to generate a container image using the source archive file retrieved from the network address.

14 . The system of claim 13 , wherein the operations further comprise:

determining, for another source archive file, that a respective size of the another source archive file is less than the threshold size;

copying, based at least on the respective size being less than the threshold size, the another source archive file to an intermediate work folder;

generating another command line to copy the another source archive file from the intermediate work folder; and

sending the another command line and the another source archive file from the intermediate work folder to the container builder such that the container builder further uses the another command line and the another source archive file to generate the container image.

15 . The system of claim 13 , wherein the operations further comprise classifying a plurality of source archive files into respective source archive types, the source archive types including at least one of a local source archive type, a remote source archive type, a Git source archive type, or a custom source archive type.

16 . The system of claim 13 , wherein the operations further comprise:

determining permissions data corresponding to the source archive file; and

storing the permissions data;

wherein the generating the one or more command lines comprises using an HTTP template configured to cause the permissions data to be retrieved for inclusion in the one or more command lines.

17 . The system of claim 16 , wherein the operations further comprise:

computing a checksum value corresponding to the source archive file during a first build of the container image;

comparing at least one of a current size or a current update time corresponding to the source archive file to a stored size or a stored update time corresponding to the source archive file; and

performing one of:

updating the checksum value corresponding to the source archive file for a second build based at least on at least one of the current size being greater than the stored size or the current update time being different from the stored update time; or

maintaining the checksum value corresponding to the source archive file for the second build based at least on at least one of the current size being equal to the stored size or the current update time being equal to the stored update time,

wherein the HTTP template is configured to include the checksum value in the one or more command lines.

18 . The system of claim 17 , wherein the checksum value, the stored size, and the stored update time are included in a metadata file corresponding to the source archive file.

19 . The system of claim 13 , wherein the system is comprised in at least one of:

a system for performing simulation operations;

a system for performing deep learning operations;

a system implemented using an edge device;

a system incorporating one or more virtual machines (VMs);

a system implemented at least partially in a data center; or

a system implemented at least partially using cloud computing resources.

20 . A method comprising:

classifying a source archive file based at least on one or more attributes associated with the source archive file;

based at least on the source archive file being classified as a local file, determining, based at least on comparing the source archive file to a threshold size, that the source archive file has a size greater than the threshold size;

configuring, based at least on the size being greater than the threshold size, a local HTTP server to provide access to the source file at a network address;

generating, using an HTTP template, one or more command lines that are executable to cause the source archive file to be retrieved from the local HTTP server using the network address; and

sending the one or more command lines to a container builder to cause the container builder to generate a container image using the source archive file retrieved from the network address.

21 . The method of claim 20 , further comprising:

determining, for another source archive file, that a respective size of the another source archive file is less than the threshold size;

based at least on the respective size being less than the threshold size, copying the another source archive file to an intermediate work folder;

generating another command line to copy the another source archive file from the intermediate work folder; and

sending the another command line and the another source archive file from the intermediate work folder to the container builder such that the container builder further uses the another command line and the another source archive file to generate the container image.

22 . The method of claim 20 , further comprising classifying a plurality of source archive files into respective source archive types, the source archive types including at least one of a local source archive type, a remote source archive type, a Git source archive type, or a custom source archive type.

23 . The method of claim 20 , further comprising:

determining permissions data corresponding to the source archive file; and

storing the permissions data;

wherein the HTTP template is configured to cause the permissions data to be retrieved for inclusion in the one or more command lines.

24 . The method of claim 20 , further comprising:

computing a checksum value corresponding to the source archive file during a first build of the container image;

comparing at least one of a current size or a current update time corresponding to the source archive file to a stored size or a stored update time corresponding to the source archive file; and

performing one of:

updating the checksum value corresponding to the source archive file for a second build based at least on at least one of the current size being greater than the stored size or the current update time being different from the stored update time; or

maintaining the checksum value corresponding to the source archive file for the second build based at least on at least one of the current size being equal to the stored size or the current update time being equal to the stored update time, wherein the HTTP template is configured to include the checksum value in the one or more command lines.

25 . The method of claim 24 , wherein the checksum value, the stored size, and the stored update time are included in a metadata file corresponding to the source archive file.

26 . The method of claim 24 , wherein the checksum value, the stored size, the stored update time are included in a metadata file in a key-value format, the metadata file associated with the source archive file.