Method and system for parallelizing backup generation operations using pseudo-asset backups
A method that is performed for backing up data. The method includes obtaining a backup request; and in response to the backup request: obtaining an asset and an asset entry from a file system metadata repository associated with the backup request; identifying asset components of the asset using the asset entry; assigning asset components to backup threads to be backed up as pseudo-assets based on the asset entry; executing the backup threads to generate pseudo-asset backups; storing the pseudo-asset backups on backup storages; and updating asset backup metadata based on the pseudo-asset backups.
1. A method for backing up data, the method comprising:
obtaining a backup request; and
in response to the backup request:
obtaining an asset and an asset entry from a file system metadata repository associated with the backup request;
identifying asset components of the asset using the asset entry, wherein the asset entry indicates a logical division of the asset components based on an asset structure, asset component sizes, and one or more exclusion tags associated with the asset specified by the asset entry;
dividing the asset components into a first portion of the asset components and a second portion of the asset components based on the logical division of the asset components, wherein the logical division of the asset components specifies an even distribution of asset component sizes between the first portion of the asset components and the second portion of the asset components;
assigning the first portion of the asset components to a first backup thread and the second portion of the asset components to a second backup thread to be backed up as pseudo-assets based on the logical division of the asset components, wherein:
the first portion of the asset components comprises a first portion of files and folders of the asset, and
the second portion of the asset components comprises a second portion of files and folders of the asset;
executing the first backup thread and the second backup thread to generate a first pseudo-asset backup and a second pseudo-asset backup, wherein the first backup thread is executed in parallel with the second backup thread, wherein an even distribution of computing resources is used to execute the first backup thread and the second backup thread;
storing the first pseudo-asset backup and the second pseudo-asset backup on backup storages;
updating asset backup metadata based on the first pseudo-asset backup and the second pseudo-asset backup; and
initiating merging of the first pseudo-asset backup and the second pseudo-asset backup on the backup storages using the asset backup metadata to generate an asset backup.
2. The method of claim 1 , wherein the asset backup comprises the asset components and the asset entry associated with the asset.
3. The method of claim 1 wherein the exclusion tags specify asset components to be excluded from being backed up using the first backup thread and the second backup thread.
4. The method of claim 1 , wherein:
the first pseudo-asset backup comprises:
a first portion of asset data of the asset corresponding to the first portion of the asset components; and
a first portion of the asset entry corresponding to the first portion of the asset data; and
the second pseudo-asset backup comprises:
a second portion of asset data of the asset corresponding to the second portion of the asset components; and
a second portion of the asset entry corresponding to the second portion of the asset data.
5. A system for backing up data, comprising:
a processor;
a backup agent, which when executed by the processor performs a method, the method comprising:
identifying asset components of the asset using the asset entry, wherein the asset entry indicates a logical division of the asset components based on an asset structure, asset component sizes, and one or more exclusion tags associated with the asset specified by the asset entry;
dividing the asset components into a first portion of the asset components and a second portion of the asset components based on the logical division of the asset components, wherein the logical division of the asset components specifies an even distribution of asset component sizes between the first portion of the asset components and the second portion of the asset components;
assigning the first portion of the asset components to a first backup thread and the second portion of the asset components to a second backup thread to be backed up as pseudo-assets based on the logical division of the asset components, wherein:
the first portion of the asset components comprises a first portion of files and folders of the asset, and
the second portion of the asset components comprises a second portion of files and folders of the asset;
executing the first backup thread and the second backup thread to generate a first pseudo-asset backup and a second pseudo-asset backup, wherein the first backup thread is executed in parallel with the second backup thread, wherein an even distribution of computing resources is used to execute the first backup thread and the second backup thread;
storing the first pseudo-asset backup and the second pseudo-asset backup on backup storages;
updating asset backup metadata based on the first pseudo-asset backup and the second pseudo-asset backup; and
initiating merging of the first pseudo-asset backup and the second pseudo-asset backup on the backup storages using the asset backup metadata to generate an asset backup.
6. The system of claim 5 , wherein the asset backup comprises the asset components and the asset entry associated with the asset.
7. The system of claim 5 , wherein the exclusion tags specify asset components to be excluded from being backed up using the first backup thread and the second backup thread.
8. The system of claim 5 , wherein:
the first pseudo-asset backup comprises:
a first portion of asset data of the asset corresponding to the first portion of the asset components; and
a first portion of the asset entry corresponding to the first portion of the asset data and
the second pseudo-asset backup comprises:
a second portion of asset data of the asset corresponding to the second portion of the asset components; and
a second portion of the asset entry corresponding to the second portion of the asset data.
9. A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method for backing up data, the method comprising:
obtaining a backup request; and
in response to the backup request:
obtaining an asset and an asset entry from a file system metadata repository associated with the backup request;
identifying asset components of the asset using the asset entry, wherein the asset entry indicates a logical division of the asset components based on an asset structure, asset component sizes, and one or more exclusion tags associated with the asset specified by the asset entry;
dividing the asset components into a first portion of the asset components and a second portion of the asset components based on the logical division of the asset components, wherein the logical division of the asset components specifies an even distribution of asset component sizes between the first portion of the asset components and the second portion of the asset components;
assigning the first portion of the asset components to a first backup thread and the second portion of the asset components to a second backup thread to be backed up as pseudo-assets based on the logical division of the asset components, wherein:
the first portion of the asset components comprises a first portion of files and folders of the asset, and
the second portion of the asset components comprises a second portion of files and folders of the asset;
executing the first backup thread and the second backup thread to generate a first pseudo-asset backup and a second pseudo-asset backup, wherein the first backup thread is executed in parallel with the second backup thread, wherein an even distribution of computing resources is used to execute the first backup thread and the second backup thread;
storing the first pseudo-asset backup and the second pseudo-asset backup on backup storages;
updating asset backup metadata based on the first pseudo-asset backup and the second pseudo-asset backup; and
initiating merging of the first pseudo-asset backup and the second pseudo-asset backup on the backup storages using the asset backup metadata to generate an asset backup.
10. The non-transitory computer readable medium of claim 9 , wherein the asset backup comprises the asset components and the asset entry associated with the asset.
11. The non-transitory computer readable medium of claim 9 , wherein the exclusion tags specify asset components to be excluded from being backed up using the first backup thread and the second backup thread.