System and method for handling implicit transactions in a hybrid cloud cache
A system for handling Implicit transactions in a hybrid cloud cache. The hybrid cloud cache maintains folders in a metadata file system that includes a hierarchy of cached folders, and maintains a meta-file containing metadata of files and folders. The system obtains a meta-file from a cloud platform and compares that to a meta-file obtained from the hybrid cloud cache. An object associated with an Implicit metadata transaction is transferred to an internal namespace of the hybrid cloud cache if the comparison indicates that certain conditions are met. The system may perform a bottom-up traversal of the namespace and may abort the traversal if an outstanding explicit transaction for the object is present or the object exists in the cloud platform. An object may be moved to a temporary storage location if there is no explicit transaction on it and the object does not exist in the cloud platform.
1. A system, comprising:
a set of computer-executable instructions; and
a processor or processors that execute the set of instructions, wherein when executed, the instructions cause the processor or processors to
obtain a copy of a meta-file from a cloud platform, the meta-file including metadata associated with a set of files and folders in a namespace of the cloud platform;
compare the copy of the meta-file obtained from the cloud platform to a copy of the meta-file obtained from a hybrid cloud cache;
transfer an object associated with an implicit metadata transaction to an orphanage of the hybrid cloud cache if the comparing indicates that:
the object is present only in the copy of the meta-file obtained from the hybrid cloud cache; or
the object is present in both the copy of the meta-file obtained from the hybrid cloud cache and the copy of the meta-file obtained from the cloud platform and a name of the object in the copy obtained from the cloud platform and in the copy obtained from the hybrid cloud cache do not match; or
the object is present only in the copy of the meta-file obtained from the cloud platform and the object is cached by the hybrid cloud cache;
conduct a bottom-up traversal of a namespace tree of the orphanage of the hybrid cloud cache;
abort the bottom-up traversal of the orphanage namespace tree if an outstanding explicit transaction for the object is present in the hybrid cloud cache or if the object still exists in the namespace of the cloud platform; and
move the object to a purgatory of the hybrid cloud cache if the outstanding explicit transaction for the object is not present and the object does not exist in the namespace of the cloud platform.
2. The system of claim 1 , wherein the set of instructions further comprise instructions that cause the processor or processors to rename the object with a unique identifier when transferring the object to the orphanage.
3. The system of claim 1 , wherein the set of instructions further comprise instructions that cause the processor or processors to reclaim the object from the orphanage if the object has moved to a new location in the cloud and the new location is cached by the hybrid cloud cache.
4. The system of claim 1 , wherein the set of instructions further comprise instructions that cause the processor or processors to leave the object in the orphanage if an outstanding explicit transaction for the object is present or the object still exists in the namespace of the cloud platform.
5. The system of claim 1 , wherein the set of instructions further comprise instructions that cause the processor or processors to conduct the bottom-up traversal of the orphanage namespace at a selected time interval.
6. The system of claim 1 , wherein the set of instructions further comprise instructions that cause the processor or processors to conduct the bottom-up traversal of the orphanage namespace in response to detecting a specific event.
7. The system of claim 1 , wherein the set of instructions further comprise instructions that cause the processor or processors to remove the object from the orphanage if all of the explicit transactions for the object have been completed by the hybrid cloud cache and the object does not exist on the cloud platform.
8. The system of claim 1 , wherein the set of instructions further comprise instructions that cause the processor or processors to classify the outstanding explicit transaction as a metadata transaction or as a data transaction.
9. The system of claim 1 , wherein the set of instructions further comprise instructions that cause the processor or processors to obtain the copy of the meta-file from the cloud platform at a selected time interval.
10. A method, comprising:
obtaining a copy of a meta-file from a cloud platform, the meta-file including metadata associated with a set of files and folders in a namespace of the cloud platform;
comparing the copy of the meta-file obtained from the cloud platform to a copy of the meta-file obtained from a hybrid cloud cache;
transferring an object associated with an implicit metadata transaction to an orphanage of the hybrid cloud cache if the comparing indicates that:
the object is present only in the copy of the meta-file obtained from the hybrid cloud cache; or
the object is present in both the copy of the meta-file obtained from the hybrid cloud cache and the copy of the meta-file obtained from the cloud platform and a name of the object in the copy obtained from the cloud platform and in the copy obtained from the hybrid cloud cache do not match; or
the object is present only in the copy of the meta-file obtained from the cloud platform and the object is cached by the hybrid cloud cache;
conducting a bottom-up traversal of a namespace tree of the orphanage of the hybrid cloud cache;
aborting the bottom-up traversal of the orphanage namespace tree if an outstanding explicit transaction for the object is present in the hybrid cloud cache or if the object still exists in the namespace of the cloud platform; and
moving the object to a purgatory of the hybrid cloud cache if the outstanding explicit transaction for the object is not present and the object does not exist in the namespace of the cloud platform.
11. The method of claim 10 , further comprising renaming the object with a unique identifier when transferring the object to the orphanage.
12. The method of claim 10 , further comprising reclaiming the object from the orphanage if the object has moved to a new location in the cloud and the new location is cached by the hybrid cloud cache.
13. The method of claim 10 , further comprising leaving the object in the orphanage if an outstanding explicit transaction for the object is present or the object still exists in the namespace of the cloud platform.
14. The method of claim 10 , further comprising conducting the bottom-up traversal of the orphanage namespace at a selected time interval or conducting the bottom-up traversal of the orphanage namespace in response to detecting a specific event.
15. The method of claim 10 , further comprising removing the object from the orphanage if all of the explicit transactions for the object have been completed by the hybrid cloud cache and the object does not exist on the cloud platform.
16. The method of claim 10 , further comprising classifying the outstanding explicit transaction as a metadata transaction or as a data transaction.
17. The method of claim 10 , further comprising fetching the copy of the meta-file from the cloud platform at a selected time interval.
18. One or more non-transitory computer-readable media including a set of computer-executable instructions, wherein when executed by a processor or processors, the instructions cause the processor or processors to perform operations comprising:
obtaining a copy of a meta-file from a cloud platform, the meta-file including metadata associated with a set of files and folders in a namespace of the cloud platform;
comparing the copy of the meta-file obtained from the cloud platform to a copy of the meta-file obtained from a hybrid cloud cache;
transferring an object associated with an implicit metadata transaction to an orphanage of the hybrid cloud cache if the comparing indicates that:
the object is present only in the copy of the meta-file obtained from the hybrid cloud cache; or
the object is present in both the copy of the meta-file obtained from the hybrid cloud cache and the copy of the meta-file obtained from the cloud platform and a name of the object in the copy obtained from the cloud platform and in the copy obtained from the hybrid cloud cache do not match; or
the object is present only in the copy of the meta-file obtained from the cloud platform and the object is cached by the hybrid cloud cache;
conducting a bottom-up traversal of a namespace tree of the orphanage of the hybrid cloud cache;
aborting the bottom-up traversal of the orphanage namespace tree if an outstanding explicit transaction for the object is present in the hybrid cloud cache or if the object still exists in the namespace of the cloud platform; and
moving the object to a purgatory of the hybrid cloud cache if the outstanding explicit transaction for the object is not present and the object does not exist in the namespace of the cloud platform.
19. The one or more computer-readable media of claim 18 , further comprising instructions which cause the processor or processors to perform one or more of renaming the object with a unique identifier when transferring the object to the orphanage, reclaiming the object from the orphanage if the object has moved to a new location in the cloud and the new location is cached by the hybrid cloud cache, and leaving the object in the orphanage if an outstanding explicit transaction for the object is present or the object still exists in the namespace of the cloud platform.
20. The one or more computer-readable media of claim 18 , further comprising instructions which cause the processor or processors to conduct the bottom-up traversal of the orphanage namespace at a selected time interval or conduct the bottom-up traversal of the orphanage namespace in response to detecting a specific event.