Data masking in a microservice architecture
A method includes retrieving an object from storage and copying the object, generating a list that identifies one or more byte ranges, of the copy of the object, to be masked, providing the list to a masker controller microservice that examines a recipe corresponding to the copy of the object, where the recipe references a slice of the copy of the object, and the slice includes one or more data segments, masking, by the masker controller microservice, a segment of the slice that is in one of the byte ranges, to create a masked segment, and replacing, in the slice, the segment with the masked segment, to create a masked slice and creating a masked object recipe that contains a reference to the masked slice, creating a masked object that includes the masked slice, and that references any unmasked segments of the slice, and deduplicating the masked object.
1. A method, comprising:
retrieving an object from storage and creating a copy of the object;
by a masker worker, generating a list that identifies one or more byte ranges, of the copy of the object, to be masked;
providing the list to a masker controller microservice;
by the masker controller microservice, examining a recipe corresponding to the copy of the object, where the recipe references a slice of the copy of the object, and the slice includes one or more data segments;
masking, by the masker controller microservice, a segment of the slice that is in one of the one or more byte ranges, to create a masked segment, and replacing, in the slice, the segment with the masked segment, to create a masked slice and creating a masked object recipe that contains a reference to the masked slice;
creating a masked object that includes the masked slice, and that references any unmasked segments of the slice; and
deduplicating the masked object.
2. The method as recited in claim 1 , wherein only masked segments of the slice are included in the masked object, and the masked object does not include any unmasked segments.
3. The method as recited in claim 1 , wherein the masked object includes a reference to an unmasked segment in storage.
4. The method as recited in claim 1 , wherein one of the one or more byte ranges spans a segment boundary.
5. The method as recited in claim 1 , wherein one of the one of more byte ranges spans a slice boundary.
6. The method as recited in claim 1 , wherein the masker worker receives the copy of the object from the masker controller microservice.
7. The method as recited in claim 1 , wherein the masker controller microservice iteratively examines each slice in the recipe to identify byte range data.
8. The method as recited in claim 1 , wherein retrieving an object from storage and creating a copy of the object are performed by the masker controller microservice.
9. The method as recited in claim 1 , further comprising generating a fingerprint of the masked segment only, and not for any unmasked segments.
10. The method as recited in claim 1 , further comprising storing the masked object in storage.
11. A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
retrieving an object from storage and creating a copy of the object;
by a masker worker, generating a list that identifies one or more byte ranges, of the copy of the object, to be masked;
providing the list to a masker controller microservice;
by the masker controller microservice, examining a recipe corresponding to the copy of the object, where the recipe references a slice of the copy of the object, and the slice includes one or more data segments;
masking, by the masker controller microservice, a segment of the slice that is in one of the one of more byte ranges, to create a masked segment, and replacing, in the slice, the segment with the masked segment, to create a masked slice;
creating a masked object that includes the masked slice, and that references any unmasked segments of the slice; and
deduplicating the masked object.
12. The non-transitory storage medium as recited in claim 11 , wherein only masked segments of the slice are included in the masked object, and the masked object does not include any unmasked segments.
13. The non-transitory storage medium as recited in claim 11 , wherein the masked object includes a reference to an unmasked segment in storage.
14. The non-transitory storage medium as recited in claim 11 , wherein one of the one of more byte ranges spans a segment boundary.
15. The non-transitory storage medium as recited in claim 11 , wherein one of the one of more byte ranges spans a slice boundary.
16. The non-transitory storage medium as recited in claim 11 , wherein the masker worker receives the copy of the object from the masker controller microservice.
17. The non-transitory storage medium as recited in claim 11 , wherein the masker controller microservice iteratively examines each slice in the recipe to identify byte range data.
18. The non-transitory storage medium as recited in claim 11 , wherein retrieving an object from storage and creating a copy of the object are performed by the masker controller microservice.
19. The non-transitory storage medium as recited in claim 11 , wherein the operations further comprise generating a fingerprint of the masked segment only, and not for any unmasked segments.
20. The non-transitory storage medium as recited in claim 11 , wherein the operations further comprise storing the masked object in storage.