File tree streaming in a virtual file system for cloud-based shared content
Systems for fast views of items in file directories or file folders when interacting with a cloud-based service platform. A server in a cloud-based environment interfaces with one or more storage devices to provide storage of shared content accessible by two or more user devices. A file tree request to view the file directory or file folder of a particular sought after item is issued from an application operating on one of the user devices. Additional file tree items in a file tree hierarchy are prefetched by the cloud-based service platform. The application closes the file tree metadata stream after receiving the portion of the file tree that pertains to the particular item and before receiving the entirety of the metadata pertaining to all of the file tree metadata of all of the items in the directory or folder that contains the particular sought after item.
1 . A method to implement prefetching in a cloud-based storage system, the method comprising:
receiving a file tree request for a lower-latency display, in a user interface, of directory information pertaining to a content object in a first directory in a hierarchical file system within a cloud-based storage system; and
in response to receiving the file tree request for the lower-latency display of the portion of the user interface,
fetching a first portion of directory metadata pertaining to the content object or the first directory from the cloud-based storage system;
predicting that the user device is to access a second directory in the hierarchical file system;
prefetching a second portion of the directory metadata for a second directory from the cloud-based storage system, the second directory different from the first directory; and
reducing latency in displaying the directory information in the user interface at least by servicing the file tree request with at least the second portion of the directory metadata, wherein prefetching reduces the latency for which at least the second directory is displayed in the user interface.
2 . The method of claim 1 , wherein
the first portion of the directory metadata corresponds to the directory,
fetching the first portion is programmatically terminated after retrieving the directory metadata pertaining to the content object but before retrieving the directory metadata for the first directory in its entirety,
the first portion of the directory metadata is retrieved in response to a navigation to the directory by a user, and
without requiring an explicit call to download additional metadata, the second portion of the directory metadata is prefetched for one or more subdirectories of the directory.
3 . The method of claim 1 , wherein prefetching is performed over a directory hierarchy with at least one of breadth-first traversal or depth-first traversal, and the second directory is a parent directory of the first directory, a child directory of the first directory, a different directory that is located at a hierarchy level at a same directory hierarchy as the first directory.
4 . The method of claim 1 , wherein prefetching is performed with prioritization such that metadata for a higher hierarchical level is prefetched before that for a lower hierarchical level.
5 . The method of claim 1 , wherein prefetching is performed with prioritization such that a call for a portion of a targeted directory tree portion will cause targeted prefetching for additional levels of the targeted directory tree portion before prefetching higher hierarchical levels.
6 . The method of claim 1 , further comprising dispatching a file tree item request in response to receipt of the file tree request, wherein prefetching the second portion of the directory metadata from the cloud-based storage system is determined by at least one of a data location, access patterns, or machine learning, or any combination thereof.
7 . The method of claim 1 , further comprising prefetching a file in a directory corresponding to the directory metadata, wherein prefetching the second portion is programmatically performed without requiring explicit calls for the second portion in the file tree request.
8 . A computer program product embodied in a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when stored in memory and executed by a processor, causes the processor to perform a set of acts for prefetching in a cloud-based storage system, the set of acts comprising:
receiving a file tree request for a lower-latency display, in a user interface, of directory information pertaining to a content object in a first directory in a hierarchical file system within a cloud-based storage system; and
in response to receiving the file tree request for the lower-latency display of the portion of the user interface,
fetching a first portion of directory metadata pertaining to the content object or the first directory from the cloud-based storage system;
predicting that the user device is to access a second directory in the hierarchical file system;
prefetching a second portion of the directory metadata for a second directory from the cloud-based storage system, the second directory different from the first directory; and
reducing latency in displaying the directory information in the user interface at least by servicing the file tree request with at least the second portion of the directory metadata, wherein prefetching reduces the latency for which at least the second directory is displayed in the user interface.
9 . The computer program product of claim 8 , wherein
the first portion of the directory metadata corresponds to the directory,
fetching the first portion is programmatically terminated after retrieving the directory metadata pertaining to the content object but before retrieving the directory metadata for the first directory in its entirety,
the first portion of the directory metadata is retrieved in response to a navigation to the directory by a user, and
without requiring an explicit call to download additional metadata, the second portion of the directory metadata is prefetched for one or more subdirectories of the directory.
10 . The computer program product of claim 8 , wherein prefetching is performed over a directory hierarchy with at least one of breadth-first traversal or depth-first traversal, and the second directory is a parent directory of the first directory, a child directory of the first directory, a different directory that is located at a hierarchy level at a same directory hierarchy as the first directory.
11 . The computer program product of claim 8 , wherein prefetching is performed with prioritization such that metadata for higher hierarchical levels are prefetched before lower hierarchical levels.
12 . The computer program product of claim 8 , wherein prefetching is performed with prioritization such that a call for a portion of a targeted directory tree will cause targeted prefetching for additional levels of the targeted subtree before prefetching higher hierarchical levels.
13 . The computer program product of claim 8 , the set of acts further comprising dispatching a file tree item request in response to receipt of the file tree request, wherein prefetching the second portion of the directory metadata from the cloud-based storage system is determined by at least one of data location, access patterns, or machine learning, or any combination thereof.
14 . The computer program product of claim 8 , further comprising prefetching a file in a directory corresponding to the directory metadata, wherein prefetching the second portion is programmatically performed without requiring explicit calls for the second portion in the file tree request.
15 . A system for prefetching in a cloud-based storage environment, the system comprising:
a storage medium having stored thereon a sequence of instructions; and
a processor or processors that execute the instructions to cause the processor or processors to perform a set of acts, the set of acts comprising:
receiving a file tree request for a lower-latency display, in a user interface, of directory information pertaining to a content object in a first directory in a hierarchical file system within a cloud-based storage system; and
in response to receiving the file tree request for the lower-latency display of the portion of the user interface,
fetching a first portion of directory metadata pertaining to the content object or the first directory from the cloud-based storage system;
predicting that the user device is to access a second directory in the hierarchical file system;
prefetching a second portion of the directory metadata for a second directory from the cloud-based storage system, the second directory different from the first directory; and
reducing latency in displaying the directory information in the user interface at least by servicing the file tree request with at least the second portion of the directory metadata, wherein prefetching reduces the latency for which at least the second directory is displayed in the user interface.
16 . The system of claim 15 , wherein
the first portion of the directory metadata corresponds to the directory,
fetching the first portion is programmatically terminated after retrieving the directory metadata pertaining to the content object but before retrieving the directory metadata for the first directory in its entirety,
the first portion of the directory metadata is retrieved in response to a navigation to the directory by a user, and
without requiring an explicit call to download additional metadata, the second portion of the directory metadata is prefetched for one or more subdirectories of the directory.
17 . The system of claim 15 , wherein prefetching is performed over a directory hierarchy with at least one of breadth-first traversal or depth-first traversal, and the second directory is a parent directory of the first directory, a child directory of the first directory, a different directory that is located at a hierarchy level at a same directory hierarchy as the first directory.
18 . The system of claim 15 , wherein prefetching is performed with prioritization such that metadata for a higher hierarchical level is prefetched before that for a lower hierarchical level, and prefetching the second portion is programmatically performed without requiring explicit calls for the second portion in the file tree request.
19 . The system of claim 15 , wherein prefetching is performed with prioritization such that a call for a portion of a targeted directory tree will cause targeted prefetching for additional levels of the targeted subtree before prefetching higher hierarchical levels.
20 . The system of claim 15 , the set of acts further comprising dispatching a file tree item request in response to receipt of the file tree request, wherein prefetching the second portion of the directory metadata from the cloud-based storage system is determined by at least one of a data location, access patterns, or machine learning, or any combination thereof.