Verification microservice for deployment of controller and worker nodes for verification paths
Described is a system for a verification microservice engine for generating and deploying a controller module and one or more worker nodes to detect corruption in a deduplicated object storage system accessible by one or more microservices while minimizing costly read operations on objects. The controller module builds local versions of slice recipe names based on metadata available object recipes. The controller module verifies the accuracy of the metadata based on whether the locally built slice recipes names match slice recipe names in object storage.
1. A computer-implemented method, comprising:
selecting, by a controller module, a similarity group verification path based on a total memory size of similarity groups in the object storage being less than a total memory size of slice recipe names in the object storage and a slice verification path based on the total memory size being greater than a total memory size of slice recipe names in the object storage;
identifying, by the controller module, one or more similarity groups to be verified by a respective worker node pre-assigned to each identified similarity group;
building, by the controller module, at least one local slice recipe name string based on object recipe metadata;
verifying, by the controller module, the accuracy of object recipe metadata that is used to generate a respective local slice recipe name based on the respective local slice recipe name matching any corresponding slice recipe name in the object storage; and
sending, by the controller module, each verified slice recipe name to a respective worker node.
2. The computer-implemented method of claim 1 , wherein verifying, by the controller module, the accuracy of object recipe metadata that is used to generate a respective local slice recipe name comprises:
detecting, by the controller module, existence of one or more object-to-slice recipe relationship corruptions when the respective local slice recipe name fails to match any slice recipe name in object storage.
3. The computer-implemented method of claim 2 , wherein a respective object-to-slice recipe relationship corruption represents corruption of at least a portion of the object metadata.
4. The computer-implemented method of claim 2 , wherein sending, by the controller module, each verified slice recipe name to a respective worker node comprises:
identifying a similarity group identifier in a respective verified slice recipe name;
identifying a worker node associated with a similarity group that corresponds with the similarity group identifier; and
sending respective verified slice recipe name to the identified worker node.
5. The computer-implemented method of claim 4 , identifying, by the controller module, one or more similarity groups to be verified by a respective worker node pre-assigned to each identified similarity group comprises:
identifying a plurality of similarity groups in object storage having a pre-assigned relationship with a particular worker node; and
sending the plurality of similarity groups to the particular worker node.
6. The computer-implemented method of claim 2 , further comprising:
collecting, by the controller module, each object-to-slice recipe relationship corruption detected by the controller module.
7. The computer-implemented method of claim 6 , further comprising:
sending, by the controller module, a request to each respective worker node for one or more object relationship type corruptions detected by each respective worker node.
8. A system comprising:
one or more processors; and
a non-transitory computer readable medium storing a plurality of instructions, which when executed, cause the one or more processors to:
select, by a controller module, a similarity group verification path based on a total memory size of similarity groups in the object storage being less than a total memory size of slice recipe names in the object storage and a slice verification path based on the total memory size being greater than a total memory size of slice recipe names in the object storage;
identify, by the controller module, one or more similarity groups to be verified by a respective worker node pre-assigned to each identified similarity group;
build, by the controller module, at least one local slice recipe name string based on object recipe metadata;
verify, by the controller module, the accuracy of object recipe metadata that is used to generate a respective local slice recipe name based on the respective local slice recipe name matching any corresponding slice recipe name in the object storage; and
send, by the controller module, each verified slice recipe name to a respective worker node.
9. The system of claim 8 , wherein verify, by the controller module, the accuracy of object recipe metadata that is used to generate a respective local slice recipe name comprises:
detect, by the controller module, existence of one or more object-to-slice recipe relationship corruptions when the respective local slice recipe name fails to match any slice recipe name in object storage.
10. The system of claim 9 , wherein a respective object-to-slice recipe relationship corruption represents corruption of the object metadata.
11. The system of claim 9 , wherein send, by the controller module, each verified slice recipe name to a respective worker node comprises:
identify a similarity group identifier in a respective verified slice recipe name;
identify a worker node associated with a similarity group that corresponds with the similarity group identifier; and
send respective verified slice recipe name to the identified worker node.
12. The system of claim 11 , identify, by the controller module, one or more similarity groups to be verified by a respective worker node pre-assigned to each identified similarity group comprises:
identify a plurality of similarity groups in object storage having a pre-assigned relationship with a particular worker node; and
send the plurality of similarity groups to the particular worker node.
13. The system of claim 9 , further comprising:
collect, by the controller module, each object-to-slice recipe relationship corruption detected by the controller module.
14. The system of claim 13 , further comprising:
send, by the controller module, a request to each respective worker node for one or more object relationship type corruptions detected by each respective worker node.
15. A computer program product comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein to be executed by one or more processors, the program code including instructions to:
select, by a controller module, a similarity group verification path based on a total memory size of similarity groups in the object storage being less than a total memory size of slice recipe names in the object storage and a slice verification path based on the total memory size being greater than a total memory size of slice recipe names in the object storage;
identify, by the controller module, one or more similarity groups to be verified by a respective worker node pre-assigned to each identified similarity group;
build, by the controller module, at least one local slice recipe name string based on object recipe metadata;
verify, by the controller module, the accuracy of object recipe metadata that is used to generate a respective local slice recipe name based on the respective local slice recipe name matching any corresponding slice recipe name in the object storage; and
send, by the controller module, each verified slice recipe name to a respective worker node.
16. The computer program product of claim 15 , wherein verify, by the controller module, the accuracy of object recipe metadata that is used to generate a respective local slice recipe name comprises:
detect, by the controller module, existence of one or more object-to-slice recipe relationship corruptions when the respective local slice recipe name fails to match any slice recipe name in object storage.
17. The computer program product of claim 16 , wherein a respective object-to-slice recipe relationship corruption represents corruption of the object metadata.
18. The computer program product of claim 16 , wherein send, by the controller module, each verified slice recipe name to a respective worker node comprises:
identify a similarity group identifier in a respective verified slice recipe name;
identify a worker node associated with a similarity group that corresponds with the similarity group identifier; and
send respective verified slice recipe name to the identified worker node.
19. The computer program product of claim 18 , identify, by the controller module, one or more similarity groups to be verified by a respective worker node pre-assigned to each identified similarity group comprises:
identify a plurality of similarity groups in object storage having a pre-assigned relationship with a particular worker node; and
send the plurality of similarity groups to the particular worker node.
20. The computer program product of claim 16 , further comprising:
collect, by the controller module, each object-to-slice recipe relationship corruption detected by the controller module; and
send, by the controller module, a request to each respective worker node for one or more object relationship type corruptions detected by each respective worker node.