Method and system for efficiently handling backup discovery operations
A method and system for efficiently handling backup discovery operations. Specifically, the disclosed method and system entail enhancing breadcrumbs—e.g., persistent files that store status and other metadata information respective to performed asset backup operations—to further include unique transaction identifiers. These transaction identifiers may serve as markers for distinguishing which subset of breadcrumbs (rather than all breadcrumbs) to forward to a central management system during asset backup discovery operations. The disclosed method and system also implement batch processing architecture to regulate resource consumption on both the client and manager sides.
1. A method for handling backup discovery operations, comprising:
receiving, from a client device manager, a first backup discovery request comprising a first last transaction identifier and a first breadcrumb paging value;
identifying, based on the first last transaction identifier, a first breadcrumb object set comprising a first plurality of breadcrumb objects;
partitioning, based on the first breadcrumb paging value, the first breadcrumb object set to obtain a first breadcrumb object subset comprising a portion of the first plurality of breadcrumb objects;
generating a first discovery batch response comprising backup operation metadata retrieved from the first breadcrumb object subset; and
transmitting, in response to the first backup discovery request, the first discovery batch response to the client device manager.
2. The method of claim 1 , wherein a transaction identifier associated with each breadcrumb object of the first plurality of breadcrumb objects reflects a numerical value greater than another numerical value representative of the first last transaction identifier.
3. The method of claim 1 , wherein a cardinality of the first breadcrumb object subset is one selected from a group consisting of less than and equal to, a numerical value representative of the first breadcrumb paging value.
4. The method of claim 1 , wherein the backup operation metadata comprises a transaction identifier associated with each breadcrumb object of the portion of the first plurality of breadcrumb objects.
5. The method of claim 4 , wherein the backup operation metadata further comprises a status recorded for an asset backup operation associated with each breadcrumb object of the portion of the first plurality of breadcrumb objects.
6. The method of claim 1 , wherein partitioning of the first breadcrumb object set further obtains a second breadcrumb object subset comprising another portion of the first plurality of breadcrumb objects.
7. The method of claim 6 , further comprising:
generating a second discovery batch response comprising backup operation metadata retrieved from the second breadcrumb object subset; and
transmitting, in response to the first backup discovery request, the second discovery batch response to the client device manager.
8. The method of claim 1 , further comprising:
prior to receiving the first backup discovery request:
receiving, from the client device manager, a first backup operation request comprising a first asset name associated with a first asset;
performing a first asset backup operation targeting the first asset to obtain a first breadcrumb object of the first plurality of breadcrumb objects;
receiving, from the client device manager, a second backup operation request comprising a second asset name associated with a second asset; and
performing a second asset backup operation targeting the second asset to obtain a second breadcrumb object of the first plurality of breadcrumb objects.
9. The method of claim 8 , wherein the first breadcrumb object comprises a first transaction identifier assigned to the first asset backup operation and the second breadcrumb object comprises a second transaction identifier assigned to the second asset backup operation.
10. The method of claim 1 , further comprising:
receiving, from the client device manager, a second backup discovery request comprising a second last transaction identifier and excluding the first breadcrumb paging value;
obtaining, based on the second backup discovery request excluding the first breadcrumb paging value, a second breadcrumb paging value stored in a discovery configuration object;
identifying, based on the second last transaction identifier, a second breadcrumb object set comprising a second plurality of breadcrumb objects;
partitioning, based on the second breadcrumb paging value, the second breadcrumb object set to obtain a second breadcrumb object subset comprising a portion of the second plurality of breadcrumb objects;
generating a second discovery batch response comprising backup operation metadata retrieved from the second breadcrumb object subset; and
transmitting, in response to the second backup discovery request, the second discovery batch response to the client device manager.
11. The method of claim 10 , wherein partitioning of the second breadcrumb object set further obtains a third breadcrumb object subset comprising another portion of the second plurality of breadcrumb objects.
12. A non-transitory computer readable medium (CRM) comprising computer readable program code, which when executed by a computer processor, enables the computer processor to:
receive, from a client device manager, a first backup discovery request comprising a first last transaction identifier and a first breadcrumb paging value;
identify, based on the first last transaction identifier, a first breadcrumb object set comprising a first plurality of breadcrumb objects;
partition, based on the first breadcrumb paging value, the first breadcrumb object set to obtain a first breadcrumb object subset comprising a portion of the first plurality of breadcrumb objects;
generate a first discovery batch response comprising backup operation metadata retrieved from the first breadcrumb object subset; and
transmit, in response to the first backup discovery request, the first discovery batch response to the client device manager.
13. The non-transitory CRM of claim 12 , wherein a transaction identifier associated with each breadcrumb object of the first plurality of breadcrumb objects reflects a numerical value greater than another numerical value representative of the first last transaction identifier.
14. The non-transitory CRM of claim 12 , wherein a cardinality of the first breadcrumb object subset is one selected from a group consisting of less than and equal to, a numerical value representative of the first breadcrumb paging value.
15. The non-transitory CRM of claim 12 , wherein the backup operation metadata comprises a transaction identifier associated with each breadcrumb object of the portion of the first plurality of breadcrumb objects.
16. The non-transitory CRM of claim 15 , wherein the backup operation metadata further comprises a status recorded for an asset backup operation associated with each breadcrumb object of the portion of the first plurality of breadcrumb objects.
17. The non-transitory CRM of claim 12 , wherein partitioning of the first breadcrumb object set further obtains a second breadcrumb object subset comprising another portion of the first plurality of breadcrumb objects.
18. The non-transitory CRM of claim 17 , comprising computer readable program code, which when executed by the computer processor, further enables the computer processor to:
generate a second discovery batch response comprising backup operation metadata retrieved from the second breadcrumb object subset; and
transmit, in response to the first backup discovery request, the second discovery batch response to the client device manager.
19. The non-transitory CRM of claim 12 , comprising computer readable program code, which when executed by the computer processor, further enables the computer processor to:
receive, from the client device manager, a second backup discovery request comprising a second last transaction identifier and excluding the first breadcrumb paging value;
obtain, based on the second backup discovery request excluding the first breadcrumb paging value, a second breadcrumb paging value stored in a discovery configuration object;
identify, based on the second last transaction identifier, a second breadcrumb object set comprising a second plurality of breadcrumb objects;
partition, based on the second breadcrumb paging value, the second breadcrumb object set to obtain a second breadcrumb object subset comprising a portion of the second plurality of breadcrumb objects;
generate a second discovery batch response comprising backup operation metadata retrieved from the second breadcrumb object subset; and
transmit, in response to the second backup discovery request, the second discovery batch response to the client device manager.
20. A system, comprising:
a client device manager comprising a first computer processor;
a client device operatively connected to the client device manager and comprising a second computer processor; and
a backup agent executing on the second computer processor and maintaining at least a plurality of breadcrumb objects,
wherein the backup agent is programmed to:
receive, from the client device manager, a backup discovery request comprising a last transaction identifier and a breadcrumb paging value;
identify, based on the last transaction identifier, a breadcrumb object set comprising the plurality of breadcrumb objects;
partition, based on the breadcrumb paging value, the breadcrumb object set to obtain a breadcrumb object subset comprising a portion of the plurality of breadcrumb objects;
generate a discovery batch response comprising backup operation metadata retrieved from the breadcrumb object subset; and
transmit, in response to the backup discovery request, the discovery batch response to the client device manager.