System and method for removal of data and metadata using an enumerator
A data storage system includes a first region, a second region, and a global enumerator. The first region stores a first copy of data. The second region stores a second copy of the data. The global enumerator performs a data control analysis of the data to identify that the first region is a high regulation region and the second region is a low regulation region; in response to making the identification: updates first metadata associated with the first copy of the data to regulate access to the first copy of the data; and updates second metadata associated with the second copy of the data to deregulate access to the second copy of the data.
1. A data storage system, comprising:
a first region comprising a first physical storage device for storing a first copy of data;
a second region comprising a second physical storage device for storing a second copy of the data;
a processor comprising circuitry;
memory; and
a global enumerator, executing using the processor and the memory, and programmed to:
perform a data control analysis of the data, using a plurality of requirements specified in a plurality of data regulation policies, to identify that the first region is a high regulation region and the second region is a low regulation region based at least in part on the plurality of data regulation policies specifying more restrictions for the first region than for the second region;
in response to making the identification:
update first metadata associated with the first copy of the data to regulate access to the first copy of the data, wherein:
the first metadata is updated by sending a first update request to cause a first modification of at least a portion of the first metadata to add a first data access requirement for accessing the first copy of the data; and
update second metadata associated with the second copy of the data to deregulate access to the second copy of the data, wherein:
the second metadata is updated by sending a second update request to cause a second modification of at least a portion of the second metadata to remove a second data access requirement for accessing the second copy of the data.
2. The data storage system of claim 1 , wherein the first copy of the data is stored as a first plurality of data chunks and the second copy of the data is stored as a second plurality of data chunks.
3. The data storage system of claim 2 , wherein the first metadata comprises:
first object level metadata associated with the first plurality of data chunks, and
first chunk level metadata associated with respective data chunks of the first plurality of data chunks.
4. The data storage system of claim 1 , wherein updating the first metadata comprises:
modifying object level metadata associated with the first copy of the data to indicate that an audit trail for the first copy of the data is to be generated.
5. The data storage system of claim 4 , wherein updating the second metadata comprises:
modifying object level metadata associated with the second copy of the data to indicate that an audit trail for the second copy of the data is not to be generated.
6. The data storage system of claim 1 , wherein updating the first metadata comprises:
modifying object level metadata associated with the first copy of the data to indicate that access to the first copy of the data is restricted.
7. The data storage system of claim 6 , wherein updating the second metadata comprises:
modifying object level metadata associated with the second copy of the data to indicate that access to the second copy of the data is unrestricted.
8. The data storage system of claim 1 , wherein the global enumerator is further programmed to:
perform a data regulation compliance analysis of the data to identify that the second region is a noncompliant region;
in response to identifying that the second region is the noncompliant region:
identify a portion of the second copy of the data that is out of compliance with data regulation policies; and
remove the portion of the second copy of the data while retaining object level metadata and chunk level metadata associated with the portion of the second copy of the data.
9. The data storage system of claim 8 , wherein removing the portion of the second copy of the data comprises:
identifying a plurality of data chunks associated with the portion of the second copy of the data; and
deleting the plurality of the data chunks.
10. The data storage system of claim 9 , wherein performing the data regulation compliance analysis comprises:
identifying, using the object level metadata, a geographic location in which a storage that stores the plurality of data chunks resides; and
comparing the geographic location to data regulation policies that specify where the data is allowed to be stored.
11. A method for managing a data storage system, comprising:
performing a data control analysis of data, using a plurality of requirements specified in a plurality of data regulation policies, to identify that a first region is a high regulation region and a second region is a low regulation region based at least in part on the plurality of data regulation policies specifying more restrictions for the first region than for the second region, wherein the first region stores a first copy of the data and the second region stores a second copy of the data; and
in response to making the identification:
updating first metadata associated with the first copy of the data to regulate access to the first copy of the data, wherein:
the first metadata is updated by sending a first update request to cause a first modification of at least a portion of the first metadata to add a first data access requirement for accessing the first copy of the data; and
updating second metadata associated with the second copy of the data to deregulate access to the second copy of the data, wherein:
the second metadata is updated by sending a second update request to cause a second modification of at least a portion of the second metadata to remove a second data access requirement for accessing the second copy of the data.
12. The method of claim 11 , wherein the first copy of the data is stored as a first plurality of data chunks and the second copy of the data is stored as a second plurality of data chunks.
13. The method of claim 12 , wherein the first metadata comprises:
first object level metadata associated with the first plurality of data chunks, and
first chunk level metadata associated with respective data chunks of the first plurality of data chunks.
14. The method of claim 11 , wherein updating the first metadata comprises:
modifying object level metadata associated with the first copy of the data to indicate that an audit trail for the first copy of the data is to be generated.
15. The method of claim 14 , wherein updating the second metadata comprises:
modifying object level metadata associated with the second copy of the data to indicate that an audit trail for the second copy of the data is not to be generated.
16. A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method for managing a data storage system, the method comprising:
performing a data control analysis of data, using a plurality of requirements specified in a plurality of data regulation policies, to identify that a first region is a high regulation region and a second region is a low regulation region based at least in part on the plurality of data regulation policies specifying more restrictions for the first region than for the second region, wherein the first region stores a first copy of the data and the second region stores a second copy of the data; and
in response to making the identification:
updating first metadata associated with the first copy of the data to regulate access to the first copy of the data, wherein:
the first metadata is updated by sending a first update request to cause a first modification of at least a portion of the first metadata to add a first data access requirement for accessing the first copy of the data; and
updating second metadata associated with the second copy of the data to deregulate access to the second copy of the data, wherein:
the second metadata is updated by sending a second update request to cause a second modification of at least a portion of the second metadata to remove a second data access requirement for accessing the second copy of the data.
17. The non-transitory computer readable medium of claim 16 , wherein the first copy of the data is stored as a first plurality of data chunks and the second copy of the data is stored as a second plurality of data chunks.
18. The non-transitory computer readable medium of claim 17 , wherein the first metadata comprises:
first object level metadata associated with the first plurality of data chunks, and
first chunk level metadata associated with respective data chunks of the first plurality of data chunks.
19. The non-transitory computer readable medium of claim 16 , wherein updating the first metadata comprises:
modifying object level metadata associated with the first copy of the data to indicate that an audit trail for the first copy of the data is to be generated.
20. The non-transitory computer readable medium of claim 19 , wherein updating the second metadata comprises:
modifying object level metadata associated with the second copy of the data to indicate that an audit trail for the second copy of the data is not to be generated.