System and method for parallel storage transformation
A transformation service is provided for a plurality of transformation jobs associated with one or more storage containers. A single transformation user runs enumeration tasks associated with data stored in the one or more storage containers. When two or more of the plurality of transformation jobs are active during a time interval, the single transformation user runs an enumeration task for all of the active transformation jobs during said time interval.
1. A method comprising:
performing a transformation service for a plurality of transformation jobs associated with one or more storage containers; and
running, by a single transformation user, enumeration tasks associated with data stored in the one or more storage containers, wherein the single transformation user is created when a first transformation job of the plurality of transformation jobs starts, and wherein the single transformation user is granted rights to query content for the one or more storage containers necessary for transforming the plurality of transformation jobs,
wherein, when two or more of the plurality of transformation jobs are active during a time interval, the single transformation user runs an enumeration task for all of the active transformation jobs during said time interval, and wherein the enumeration task is shared among the active transformation jobs during the time interval.
2. The method of claim 1 further comprising:
when one of the transformation jobs is started for a storage under transformation, creating the transformation user if there are no other active transformation jobs, and granting the transformation user rights to query content for all of the storage containers necessary for transforming the one transformation job.
3. The method of claim 1 further comprising:
when one of the plurality of transformation jobs is finished for a storage under transformation, and if there are unique storage containers to transform for the one transformation job, revoking the rights to query content for all unique containers to transform for the one transformation job, wherein the transformation user maintains rights for storage containers of the one transformation job shared with other transformation jobs.
4. The method of claim 1 wherein when the transformation job is finished for a storage under transformation, and if there are no other transformation jobs that are active, the method further comprises deleting the transformation user.
5. The method of claim 1 wherein each run enumeration task results in an object with related information regarding a container identification identifying the transformation jobs concerned with the object, and the container identification and identified jobs identify target containers for the object, and wherein metadata associated with the object is read and stored in an object index a number of N times, N being the number of identified target containers.
6. The method of claim 1 wherein the transformation user runs an enumeration task unique to one or more storage containers of a first transformation job during a first time interval, the transformation user runs an enumeration task unique to one or more storage containers of a second transformation job during a second time interval, and the transformation user runs an enumeration task for one or more storage containers of both the first and second transformation jobs during a third time interval in which both the first and second transformation jobs are active.
7. A data processing system comprising:
electronic hardware circuitry constructed to:
perform a transformation service for a plurality of transformation jobs associated with one or more storage containers; and
run, by a single transformation user, enumeration tasks associated with data stored in the one or more storage containers, wherein the single transformation user is created when a first transformation job of the plurality of transformation jobs starts, and wherein the single transformation user is granted rights to query content for the one or more storage containers necessary for transforming the plurality of transformation jobs,
wherein, when two or more of the plurality of transformation jobs are active during a time interval, the single transformation user runs an enumeration task for all of the active transformation jobs during said time interval, and wherein the enumeration task is shared among the active transformation jobs during the time interval.
8. The data processing system of claim 7 wherein the electronic hardware circuitry is further constructed to:
when one of the transformation jobs is started for a storage under transformation, create the transformation user if there are no other active transformation jobs, and grant the transformation user rights to query content for all of the storage containers necessary for transforming the one transformation job.
9. The data processing system of claim 7 wherein the electronic hardware circuitry is further constructed to:
when one of the plurality of transformation jobs is finished for a storage under transformation, and if there are unique storage containers to transform for the one transformation job, revoke the rights to query content for all unique containers to transform for the one transformation job, wherein the transformation user maintains rights for storage containers of the one transformation job shared with other transformation jobs.
10. The data processing system of claim 7 wherein when the transformation job is finished for a storage under transformation, and if there are no other transformation jobs that are active, the electronic hardware circuitry is further constructed to delete the transformation user.
11. The data processing system of claim 7 wherein each run enumeration task results in an object with related information regarding a container identification identifying the transformation jobs concerned with the object, and the container identification and identified jobs identify target containers for the object, and wherein metadata associated with the object is read and stored in an object index a number of N times, N being the number of identified target containers.
12. The data processing system of claim 7 wherein the transformation user runs an enumeration task unique to one or more storage containers of a first transformation job during a first time interval, the transformation user runs an enumeration task unique to one or more storage containers of a second transformation job during a second time interval, and the transformation user runs an enumeration task for one or more storage containers of both the first and second transformation jobs during a third time interval in which both the first and second transformation jobs are active.
13. A non-transitory computer-readable storage medium storing computer-executable instructions, the instructions causing a machine to execute a process comprising:
performing a transformation service for a plurality of transformation jobs associated with one or more storage containers; and
running, by a single transformation user, enumeration tasks associated with data stored in the one or more storage containers, wherein the single transformation user is created when a first transformation job of the plurality of transformation jobs starts, and wherein the single transformation user is granted rights to query content for the one or more storage containers necessary for transforming the plurality of transformation jobs,
wherein, when two or more of the plurality of transformation jobs are active during a time interval, the single transformation user runs an enumeration task for all of the active transformation jobs during said time interval, and wherein the enumeration task is shared among the active transformation jobs during the time interval.
14. The non-transitory computer-readable storage medium of claim 13 wherein the process further comprises:
when one of the transformation jobs is started for a storage under transformation, creating the transformation user if there are no other active transformation jobs, and granting the transformation user rights to query content for all of the storage containers necessary for transforming the one transformation job.
15. The non-transitory computer-readable storage medium of claim 13 wherein the process further comprises:
when one of the plurality of transformation jobs is finished for a storage under transformation, and if there are unique storage containers to transform for the one transformation job, revoking the rights to query content for all unique containers to transform for the one transformation job, wherein the transformation user maintains rights for storage containers of the one transformation job shared with other transformation jobs.
16. The non-transitory computer-readable storage medium of claim 13 wherein when the transformation job is finished for a storage under transformation, and if there are no other transformation jobs that are active, the process further comprises deleting the transformation user.
17. The non-transitory computer-readable storage medium of claim 13 wherein each run enumeration task results in an object with related information regarding a container identification identifying the transformation jobs concerned with the object, and the container identification and identified jobs identify target containers for the object, and wherein metadata associated with the object is read and stored in an object index a number of N times, N being the number of identified target containers.
18. The non-transitory computer-readable storage medium of claim 13 wherein the transformation user runs an enumeration task unique to one or more storage containers of a first transformation job during a first time interval, the transformation user runs an enumeration task unique to one or more storage containers of a second transformation job during a second time interval, and the transformation user runs an enumeration task for one or more storage containers of both the first and second transformation jobs during a third time interval in which both the first and second transformation jobs are active.