Resynchronizing metadata in a content management system
A client can resynchronize tree data structures representing a server, local, and sync state of content items in an account. In some aspects, the client identifies a local set of content items on the account and generates a local tree representing a filesystem state and including a local node for each item in the local set. The client determines a server state for a remote set of content items on the account. Based on the server state, the client generates a remote tree representing the remote set and including a respective node for each item in the remote set. Based on the local and remote trees, the client generates a sync tree including matching nodes between the local and remote trees, the sync tree representing a synchronized state between the filesystem state and the server state. The client then stores the local, remote, and sync trees.
1. A method comprising:
identifying, via a client device, a local set of content items stored at the client device;
based on the local set of content items, generating, via the client device, a local tree data structure representing a filesystem state at the client device,
receiving, from a content management system, a server state identifying a remote set of content items stored at the content management system;
generating, via the client device, a remote tree data structure representing the remote set of content items based on a received server state; and
based on the local tree data structure and the remote tree data structure, generating, via the client device, a sync tree data structure, the sync tree data structure representing a synchronized state between the filesystem state and the received server state.
2. The method of claim 1 , wherein identifying the local set of content items stored at the client device comprises:
crawling a storage at the client device to identify the local set of content items at the client device;
indexing the local set of content items; and
calculating a respective hash value for each of the local set of content items.
3. The method of claim 2 , wherein each respective local node in the local tree data structure comprises the respective hash value calculated for a corresponding local content item in the local set of content items and respective metadata associated with the corresponding local content item.
4. The method of claim 3 , wherein the respective metadata comprises at least one of a respective file identifier, a respective directory identifier, or a respective content item name.
5. The method of claim 4 , wherein each respective remote node in the remote tree data structure comprises a second respective hash value calculated for a corresponding remote content item in the remote set of content items and respective metadata associated with the corresponding remote content item.
6. The method of claim 1 , wherein generating the sync tree data structure comprises:
a respective local node corresponding to each of the local set of content items and for each respective local node in the local tree data structure:
determining whether the respective local node matches one of a plurality of remote nodes in the remote tree data structure; and
adding the respective local node to the sync tree data structure when the respective local node matches one of the plurality of remote nodes in the remote tree data structure.
7. The method of claim 6 , further comprising excluding the respective local node from the sync tree data structure when the respective local node does not match one of the plurality of remote nodes in the remote tree data structure.
8. The method of claim 7 , wherein determining whether the respective local node matches one of the plurality of remote nodes in the remote tree data structure comprises searching the remote tree data structure for at least one of a respective file identifier associated with the respective local node, a respective directory identifier associated with the respective local node, or a respective content item name associated with the respective local node.
9. The method of claim 8 , further comprising:
prior to excluding the respective local node from the sync tree data structure when the respective local node does not match one of the plurality of remote nodes in the remote tree data structure, querying a journal of revisions at the content management system for a record comprising at least one of the respective file identifier associated with the respective local node, the respective directory identifier associated with the respective local node, or the respective content item name associated with the respective local node; and
excluding the respective local node from the sync tree data structure only when the content management system indicates that the journal of revisions does not contain the record comprising at least one of the respective file identifier, the respective directory identifier, or the respective content item name.
10. The method of claim 1 , further comprising:
identifying a difference between the sync tree data structure and the remote tree data structure, the difference representing one or more conflicts between the server state and the filesystem state;
converging the server state and the filesystem state to yield a resynchronized state; and
synchronizing the local tree data structure, the sync tree data structure, and the remote tree data structure based on the resynchronized state.
11. A non-transitory computer readable medium comprising instructions, the instructions, when executed by a computing device, cause the computing device to:
identify a local set of content items;
based on the local set of content items, generate a local tree data structure representing a filesystem state at the computing device;
receive, from a content management system, a server state identifying a remote set of content items stored at the content management system;
generate a remote tree data structure representing a remote set of content items based on the server state; and
based on the local tree data structure and the remote tree data structure, generate a sync tree data structure, the sync tree data structure representing a synchronized state between the filesystem state and the received server state.
12. The non-transitory computer readable medium of claim 11 , wherein identifying the local set of content items comprises:
crawling a storage at the computing device to identify the local set of content items;
indexing the local set of content items; and
calculating a respective hash value for each corresponding local content item in the local set of content items.
13. The non-transitory computer readable medium of claim 12 , wherein a respective local node corresponds to each of the local set of content items and each respective local node in the local tree data structure comprises the respective hash value calculated for the corresponding local content item and respective metadata associated with the corresponding local content item, wherein the respective metadata comprises at least one of a respective file identifier, a respective directory identifier, or a respective content item name.
14. The non-transitory computer readable medium of claim 11 , wherein generating the sync tree data structure comprises:
a respective local node corresponding to each of the local set of content items and for each respective local node in the local tree data structure:
determining whether the respective local node matches one of a plurality of remote nodes in the remote tree data structure; and
adding the respective local node to the sync tree data structure when the respective local node matches one of the plurality of remote nodes in the remote tree data structure.
15. The non-transitory computer readable medium of claim 14 , storing instructions which, when executed by the computing device, cause the computing device to:
when the respective local node does not match one of the plurality of remote nodes in the remote tree data structure, query a journal of revisions at the content management system for a record comprising at least one of a respective file identifier associated with the respective local node, a respective directory identifier associated with the respective local node, or a respective content item name associated with the respective local node; and
exclude the respective local node from the sync tree data structure when the content management system indicates that the journal of revisions does not contain the record comprising at least one of the respective file identifier, the respective directory identifier, or the respective content item name.
16. The non-transitory computer readable medium of claim 11 , storing instructions which, when executed by the computing device, cause the computing device to:
identify a difference between the sync tree data structure and the remote tree data structure, the difference representing one or more differences between the server state and the filesystem state;
converge the server state and the filesystem state to yield a resynchronized state; and
based on the resynchronized state, synchronize the local tree data structure, the sync tree data structure, and the remote tree data structure by modifying at least one of the local tree data structure, the sync tree data structure, or the remote tree data structure.
17. A computing device comprising:
one or more processors; and
at least one non-transitory computer readable medium having stored therein instructions which, when executed by the one or more processors, cause the computing device to:
identify a local set of content items;
based on the local set of content items, generate a local tree data structure representing a filesystem state at the computing device;
receive, from a content management system, a remote state corresponding to a remote set of content items on the content management system;
generate a remote tree data structure representing a remote set of content items based on a received server state; and
based on the local tree data structure and the remote tree data structure, generate a sync tree data structure, the sync tree data structure representing a synchronized state between the filesystem state and the remote state.
18. The computing device of claim 17 , storing instructions which, when executed by the one or more processors, cause the computing device to:
identify a difference between the sync tree data structure and the remote tree data structure, the difference representing one or more differences between the remote state and the filesystem state;
converge the remote state and the filesystem state to yield a resynchronized state; and
based on the resynchronized state, synchronize the local tree data structure, the sync tree data structure, and the remote tree data structure by modifying at least one of the local tree data structure, the sync tree data structure, or the remote tree data structure.
19. The computing device of claim 17 , wherein identifying the local set of content items comprises:
crawling the at least one non-transitory computer readable medium to identify the local set of content items;
indexing the local set of content items; and
calculating a respective hash value for each corresponding local content item in the local set of content items.
20. The computing device of claim 19 , wherein each respective local node in the local tree data structure comprises the respective hash value calculated for the corresponding local content item and respective metadata comprising at least one of a respective file identifier, a respective directory identifier, or a respective content item name, wherein generating the sync tree data structure comprises:
for each respective local node in the local tree data structure:
determining whether the respective local node matches one of a plurality of remote nodes in the remote tree data structure;
adding the respective local node to the sync tree data structure when the respective local node matches one of the plurality of remote nodes in the remote tree data structure; and
excluding the respective local node from the sync tree data structure when the respective local node does not match one of the plurality of remote nodes in the remote tree data structure and a journal of revisions at the content management system does not contain a journal entry comprising at least one of the respective file identifier, the respective directory identifier, or the respective content item name.