Metadata peering with improved inodes
View Patent ↗Example apparatus and methods concern metadata peering that allows a file system to address a file in a local namespace while actually accessing the file using an address used by a shared secondary storage. This allows the file system to operate unmodified and more efficiently in a native mode that addresses files in the shared secondary storage using addresses local to a computer running the file system. An example method accesses an inode associated with a file that is stored in a shared secondary storage. The method accesses a local namespace identifier used by the file system to access the file and accesses a reference used by the shared secondary storage to access the file. The reference is not associated with the local namespace. The method stores an opaque value in the inode. The opaque value facilitates accessing the file in the shared secondary storage using the local namespace.
1. An apparatus, comprising:
a processor;
a memory that stores a data structure used by a file system running on the apparatus to access a file stored in shared secondary storage, where the file is managed by the file system;
a set of circuits; and
an interface that connects the processor, the memory, and the set of circuits;
the set of circuits comprising:
a first circuit that acquires first locating data associated with the file, where the first locating data is local to the file system and where the first locating data is used by the file system for addressing the file;
a second circuit that acquires second locating data associated with the file, where the second locating data is local to the shared secondary storage, and where the second locating data is used by the shared secondary storage for addressing the file,
a third circuit that stores an opaque value in the data structure, where the opaque value relates the first locating data to the second locating data and relates two different addresses of the file system and the shared secondary storage as two disparate file systems, and wherein the opaque value enables an indirect arrival to the file in the shared secondary storage without providing a direct visibility to either of the two disparate file systems; and
a fourth circuit that detects a request by the file system to access the file using the first locating data, where the fourth circuit is configured to control the apparatus to retrieve the opaque value associated with the first locating data and to provide a request to the shared secondary storage, where the request to the shared secondary storage includes the opaque value.
2. The apparatus of claim 1 , where the first locating data comprises a length and an offset associated with a data store local to the file system.
3. The apparatus of claim 1 , comprising a fifth circuit configured to control the apparatus to provide at least a portion of the data structure to an external data store.
4. The apparatus of claim 3 , where the fifth circuit is configured to control the apparatus to provide a transaction to the external data store, where the transaction includes information about the request by the file system to access the file using the first locating data and where the transaction includes information about a result of the request to the shared secondary storage.
5. The apparatus of claim 4 , comprising a sixth circuit configured to control the apparatus to rebuild a portion of the file system using data stored in the external data store.
6. The apparatus of claim 5 , where controlling the apparatus to rebuild the portion of the file system comprises:
accessing a selected data structure in the external data store, where the selected data structure is associated with a file to be rebuilt in the portion of the file system;
initializing a rebuild data structure associated with the file to be rebuilt, where the rebuild data structure is initialized using a first opaque value from the selected data structure, where the first opaque value relates a first locating data associated with the file to be rebuilt to a second locating data associated with the file to be rebuilt, where the first locating data associated with the file to be rebuilt is used by the file system to address the file to be rebuilt, and where the second locating data associated with the file to be rebuilt is used by the secondary storage to address the file to be rebuilt;
accessing a selected transaction associated with the file to be rebuilt, where the selected transaction is stored in the external data store, and where the selected transaction includes information about previous accesses to the file to be rebuilt, and
updating the rebuild data structure based, at least in part, on the selected transaction.
7. The apparatus of claim 6 , comprising:
selectively updating the rebuild data structure with a current first locating data associated with the file to be rebuilt or with a current opaque value.
8. The apparatus of claim 4 , comprising a seventh circuit configured to control the apparatus to build a second file system using data stored in the external data store.
9. The apparatus of claim 7 , where controlling the apparatus to build the second file system comprises:
accessing a selected data structure in the external data store, where the selected data structure is associated with a selected file to be included in the second file system;
initializing a build data structure associated with the selected file, where the build data structure is initialized using a first opaque value from the selected data structure, where the first opaque value relates a first locating data associated with the selected file to a second locating data associated with the selected file, where the first locating data associated with the selected file is used by the second file system to address the selected file, and where the second locating data associated with the selected file is used by the secondary storage to address the selected file;
accessing a selected transaction associated with the selected file, where the selected transaction is stored in the external data store, and where the selected transaction includes information about previous accesses to the selected file, and
updating the build data structure based, at least in part, on the selected transaction.
10. The apparatus of claim 2 , where the opaque value comprises a value that resolves to an actual address on a data storage device, the actual address comprising a length and an offset field.
11. The apparatus of claim 10 , where each of the disparate file systems comprises an associated namespace, and includes access to a shared location of the shared secondary storage, where the file system comprises an associated local namespace and local view.
12. The apparatus of claim 11 , where the shared secondary storage comprises a plurality of disk drives, a plurality of tape drives, a cloud based storage, or a plurality of other data stores comprising solid state devices, where a single copy of data is stored in the shared secondary storage with different references of the disparate file systems, respectively leading to the shared location of the shared secondary storage.
13. The apparatus of claim 12 , where indirection information enables using the different references to access the file in the shared secondary storage even though the file system accesses the file using the local namespace identifier.