Systems and methods for distributed system scanning
View Patent ↗Systems and methods are provided for scanning files and directories in a distributed file system on a network of nodes. The nodes include metadata with attribute information corresponding to files and directories distributed on the nodes. In one embodiment, the files and directories are scanned by commanding the nodes to search their respective metadata for a selected attribute. At least two of the nodes are capable of searching their respective metadata in parallel. In one embodiment, the distributed file system commands the nodes to search for metadata data structures having location information corresponding to a failed device on the network. The metadata data structures identified in the search may then be used to reconstruct lost data that was stored on the failed device.
1. A method for handling a failure in a distributed file system, the method comprising:
for at least one file of a plurality of files, storing a the file across one or more memory devices of a plurality of nodes in a network, at least some of the files distributed in blocks across multiple nodes;
creating a data block location information structure associated with the file and listing, for all of the blocks in the file, the memory devices and physical addresses where the blocks are stored;
creating a consolidated device list which is associated with the file and which comprises an abbreviated listing of each of the one or more memory devices that are listed in the data block location information structure, but which does not include a duplicate listing of a particular memory device for all of the blocks in the file that are stored on the particular memory device, wherein the device list is separate from and includes less information than the data block location information structure;
storing metadata data structures across the plurality of nodes, at least two of the metadata data structures stored on different nodes, each of the metadata data structures referencing information about a corresponding one of the plurality of files, and the metadata data structures comprising the data block location information structure and the consolidated device list of the corresponding file;
detecting that one of the memory devices in the network has failed;
using one or more processors, commanding the nodes to search each consolidated device list stored in each of their respective metadata data structures to determine whether the failed memory device is used to store information referenced by their respective metadata data structures, wherein searching each consolidated device list allows the nodes to make the determination without searching through the data block location information structure, which includes more information than the consolidated device list; and
receiving an indication from the nodes of which metadata data structures reference information stored on the failed device.
2. The method of claim 1 , further comprising commanding the nodes to access the identified metadata data structures to reconstruct the information stored on the failed device.
3. The method of claim 1 , wherein the files comprise files and directories.
4. The method of claim 1 , wherein at least some of the nodes perform at least a portion of the search of the consolidated lists of their respective metadata data structures in parallel with at least a portion of the search of one or more other nodes.
5. The method of claim 1 , wherein accessing the identified metadata data structures comprises reading pointers to at least one of file content blocks, metadata data blocks, and protection data blocks stored on the failed memory device.
6. The system of claim 1 , wherein the consolidated device list includes a single reference to each of the one or more memory devices that is listed in the data block location information structure, regardless of the number of blocks of the associated file that are stored on the particular memory device.
7. A distributed file system, comprising:
a plurality of nodes in a network, each comprising:
one or more processors; and
at least one memory device, wherein:
a plurality of files is stored across the memory devices of the plurality of nodes, at least some of the files distributed in blocks across multiple nodes, and
metadata data structures are stored across the memory devices of the plurality of nodes, at least two of the metadata data structures stored on different nodes, each of the metadata data structures referencing information about a corresponding one of the files, and the respective metadata data structures comprising:
a data block location information structure associated with the file corresponding to the respective metadata data structure and listing, for all of the blocks in the file, the memory devices and physical addresses where the blocks in the file are stored; and
a consolidated device list associated with the file corresponding to the respective metadata data structure and which comprises an abbreviated listing of each of one or more memory devices that are listed in the data block location information structure, wherein the consolidated device list does not include a duplicate listing of a particular memory device for all of the blocks in the file that are stored on the particular memory device, and is separate from and includes less information than the data block location information structure; and
wherein the one or more processors of at least one of the nodes are configured to:
detect that one of the memory devices in the network has failed;
command the nodes to search each consolidated device list stored in each of their respective metadata data structures to determine whether the failed memory device is used to store information referenced by their respective metadata data structures, wherein searching each consolidated device list allows the nodes to make the determination without searching through the data block location information structure, which includes more information than the consolidated device list; and
receive an indication from the nodes of which metadata data structures reference information stored on the failed device.
8. The system of claim 7 , wherein the one or more processors of the at least one of the nodes are further configured to command the nodes to access the identified metadata data structures to reconstruct the information stored on the failed device.
9. The system of claim 7 , wherein the files comprise files and directories.
10. The system of claim 7 , wherein at least some of the nodes perform at least a portion of the search of the consolidated lists of their respective metadata data structures in parallel with at least a portion of the search of one or more other nodes.
11. The system of claim 7 , wherein the nodes are configured to access the identified metadata data structures at least in part by reading pointers to at least one of file content blocks, metadata data blocks, and protection data blocks stored on the failed memory device.
12. The system of claim 7 , wherein the consolidated device lists each include a single reference to each one or more memory devices that is listed in the corresponding data block location information structure, regardless of the number of blocks of the associated file that are on the particular memory device.