Large folder operations in a collaborative cloud-based environment
Systems for managing operating on folders that comprise a large number of objects in a cloud-based collaboration platform. A cloud-based environment includes storage devices that store content objects accessible by collaborating users. The content objects are identified by one or more hierarchical path identifiers comprising a folder identifier corresponding to a logical grouping. The hierarchical path identifiers are stored in a relational database. Methods receive an operation command from a user to operate on the content objects associated with the logical grouping, after which a relational database search query is generated. Processing of the query returns a plurality of result sets corresponding to respective portions of the content objects associated with the logical grouping. The operation command is applied iteratively over the plurality of result sets, and the command is finalized after the last iteration. Collaborators can perform certain operations on content objects even while the iterations are in progress.
1. A method comprising:
identifying a server in a cloud-based environment that is interfaced with one or more storage devices that store one or more content objects accessible by one or more users, wherein the content objects are identified by one or more hierarchical path identifiers comprising at least one folder identifier corresponding to a logical folder, and wherein the hierarchical path identifiers are stored in a relational database in the storage devices;
receiving an operation command from at least one of the users to operate on the content objects associated with the logical folder;
generating a search query for the relational database, wherein the search query is configured to select respective portions of the content objects associated with the logical folder; and
invoking the operation command over a plurality of result sets selected from issuing the search query multiple times to partition the operation command into separate operations, wherein the separate operations are applied to different portions of the content objects by:
(a) issuing the search query to return a portion of the result sets corresponding to the respective portions of the content objects associated with the logical folder,
(b) invoking the operation command to the portion of the result sets that is returned by the search query that was issued, and
(c) repeating the steps of (a) and (b) by issuing a subsequent search query as the search query until the operation command that was received from the at least one of the users is completed, wherein the subsequent search query returns a subsequent portion of the result sets each time the search query reissues.
2. The method of claim 1 , further comprising updating status metadata to indicate an operation status for the respective portion of the content objects associated with the logical folder.
3. The method of claim 2 , further comprising repeatedly invoking the operation command on at least a portion of the result sets based at least in part on the operation status.
4. The method of claim 2 , further comprising generating a revert query for the relational database based at least in part on the operation status.
5. The method of claim 2 , wherein the operation status characterizes at least one of, a successfully completed status, or a pending completion status, or a queued status, or a failed status.
6. The method of claim 1 , further comprising updating permissibility metadata to indicate an operation permissibility for the respective portion of the content objects associated with the logical folder.
7. The method of claim 6 , wherein the operation permissibility is based at least in part on the operation command.
8. The method of claim 6 , wherein the operation permissibility characterizes an association between the operation command and one or more concurrent operations on the content objects.
9. The method of claim 6 , wherein the permissibility metadata further indicates a logical locking of at least one of, the logical folder, a parent folder of the logical folder, or at least one of the content objects associated with the logical folder.
10. The method of claim 1 , wherein the search query comprises a returned item bound to determine at least one of the result sets.
11. The method of claim 10 , wherein the returned item bound is a chunk size.
12. A computer readable medium, embodied in a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when stored in memory and executed by a processor causes the processor to perform a set of acts, the acts comprising:
identifying a server in a cloud-based environment that is interfaced with one or more storage devices that store one or more content objects accessible by one or more users, wherein the content objects are identified by one or more hierarchical path identifiers comprising at least one folder identifier corresponding to a logical folder, and wherein the hierarchical path identifiers are stored in a relational database in the storage devices;
receiving an operation command from at least one of the users to operate on the content objects associated with the logical folder;
generating a search query for the relational database, wherein the search query is configured to select respective portions of the content objects associated with the logical folder; and
invoking the operation command over a plurality of result sets selected from issuing the search query multiple times to partition the operation command into separate operations, wherein the separate operations are applied to different portions of the content objects by:
(a) issuing the search query to return a portion of the result sets corresponding to the respective portions of the content objects associated with the logical folder,
(b) invoking the operation command to the portion of the result sets that is returned by the search query that was issued, and
repeating the steps of (a) and (b) by issuing a subsequent search query as the search query until the operation command that was received from the at least one of the users is completed, wherein the subsequent search query returns a subsequent portion of the result sets each time the search query reissues.
13. The computer readable medium of claim 12 , further comprising instructions which, when stored in memory and executed by the processor causes the processor to perform acts of updating status metadata to indicate an operation status for the respective portion of the content objects associated with the logical folder.
14. The computer readable medium of claim 13 , further comprising instructions which, when stored in memory and executed by the processor causes the processor to perform acts of repeatedly invoking the operation command on at least a portion of the result sets based at least in part on the operation status.
15. The computer readable medium of claim 13 , further comprising instructions which, when stored in memory and executed by the processor causes the processor to perform acts of generating a revert query for the relational database based at least in part on the operation status.
16. The computer readable medium of claim 13 , wherein the operation status characterizes at least one of, a successfully completed status, or a pending completion status, or a queued status, or a failed status.
17. The computer readable medium of claim 12 , further comprising instructions which, when stored in memory and executed by the processor causes the processor to perform acts of updating permissibility metadata to indicate an operation permissibility for the respective portion of the content objects associated with the logical folder.
18. The computer readable medium of claim 17 , wherein the operation permissibility is based at least in part on the operation command.
19. A system comprising:
a storage medium having stored thereon a sequence of instructions; and
a processor or processors that execute the instructions to cause the processor or processors to perform a set of acts, the acts comprising,
identifying a server in a cloud-based environment that is interfaced with one or more storage devices that store one or more content objects accessible by one or more users, wherein the content objects are identified by one or more hierarchical path identifiers comprising at least one folder identifier corresponding to a logical folder, and wherein the hierarchical path identifiers are stored in a relational database in the storage devices;
receiving an operation command from at least one of the users to operate on the content objects associated with the logical folder;
generating a search query for the relational database, wherein the search query is configured to select respective portions of the content objects associated with the logical folder; and
invoking the operation command over a plurality of result sets selected from issuing the search query multiple times to partition the operation command into separate operations, wherein the separate operations are applied to different portions of the content objects by:
(a) issuing the search query to return a portion of the result sets corresponding to the respective portions of the content objects associated with the logical folder,
(b) invoking the operation command to the portion of the result sets that is returned by the search query that was issued, and
repeating the steps of (a) and (b) by issuing a subsequent search query as the search query until the operation command that was received from the at least one of the users is completed, wherein the subsequent search query returns a subsequent portion of the result sets each time the search query reissues.
20. The system of claim 19 , wherein an operation status characterizes at least one of, a successfully completed status, or a pending completion status, or a queued status, or a failed status.