Distributed storage device and data management method in distributed storage device
The number of inter-node communications in inter-node deduplication can be reduced and both performance stability and high capacity efficiency can be achieved. A storage drive of storage nodes stores files that are not deduplicated in the plurality of storage nodes, duplicate data storage files in which deduplicated duplicate data is stored, and cache data storage files in which cache data of duplicate data stored in another storage node is stored, in which when a read access request for the cache data is received, the processors of the storage nodes read the cache data if the cache data is stored in the cache data storage file, and request another storage node to read the duplicate data related to the cache data if the cache data is discarded.
1. A distributed storage device comprising a plurality of storage nodes, wherein
the storage node includes a storage device and a processor,
the plurality of storage nodes have a deduplication function for deduplication between storage nodes,
the storage device stores files that are not deduplicated in the plurality of storage nodes, duplicate data storage files in which deduplicated duplicate data is stored, and cache data storage files in which cache data of duplicate data stored in another storage node is stored,
the processor
discards the cache data when a predetermined condition is satisfied, and
when a read access request for the cache data is received, reads the cache data if the cache data is stored in the cache data storage file, and requests another storage node to read the duplicate data related to the cache data if the cache data is discarded.
2. The distributed storage device according to claim 1 , wherein
the predetermined condition is that the free space of the storage device in the storage node is small.
3. The distributed storage device according to claim 1 , wherein
the processor discards a part or all of the cache data of the cache data storage file and stores the duplicate data related to the read access request read from the other storage node in the cache data storage file.
4. The distributed storage device according to claim 3 , wherein
a predetermined plurality of files are access units from the server, the predetermined plurality of files in the access unit are distributed and stored in a plurality of storage nodes, and a distributed storage node that stores the files is determined by the storage node, and
when discarding the cache data storage file, the processor preferentially leaves the cache data of the duplicate data related to the file that is held by the own storage node in the cache data storage file.
5. The distributed storage device according to claim 3 , wherein
a predetermined plurality of files are access units from the server, the predetermined plurality of files in the access unit are distributed and stored in a plurality of storage nodes, and a distributed storage node that stores the files is determined by the storage node, and
when discarding the cache data, if the cache data to be discarded is a file that configures a part of an access unit, the processor discards the cache data of another file that configures the same access unit as the file.
6. The distributed storage device according to claim 1 , wherein
when the processor receives a write access request and detects that the data related to the write access request is duplicated with any of the data, the processor performs deduplication and stores the data related to the write access request in the cache data storage file.
7. The distributed storage device according to claim 6 , wherein
the processor discards a part or all of the cache data of the cache data storage file and stores the detected duplicate data among the data related to the write access request in the cache data storage file.
8. The distributed storage device according to claim 7 , wherein
a predetermined plurality of files are access units from the server, the predetermined plurality of files in the access unit are distributed and stored in a plurality of storage nodes, and a distributed storage node that stores the files is determined by the storage node, and
when discarding the cache data storage file, the processor preferentially leaves the cache data of the duplicate data related to the file that is held by the own storage node in the cache data storage file.
9. The distributed storage device according to claim 7 , wherein
a predetermined plurality of files are access units from the server, the predetermined plurality of files in the access unit are distributed and stored in a plurality of storage nodes, and a distributed storage node that stores the files is determined by the storage node, and
when discarding the cache data, if the cache data to be discarded is a file that configures a part of an access unit, the processor discards the cache data of another file that configures the same access unit as the file.
10. The distributed storage device according to claim 1 , wherein
when the processor receives a write access request and writes to a file that is not deduplicated, a duplication determination is performed at any timing, and when duplicate data is detected in the written data, the duplicate data is stored in the cache data storage file.
11. The distributed storage device according to claim 10 , wherein
the processor discards a part or all of the cache data of the cache data storage file and stores the detected duplicate data among the data related to the write access request in the cache data storage file.
12. The distributed storage device according to claim 11 , wherein
a predetermined plurality of files are access units from the server, the predetermined plurality of files in the access unit are distributed and stored in a plurality of storage nodes, and a distributed storage node that stores the files is determined by the storage node, and
when discarding the cache data storage file, the processor preferentially leaves the cache data of the duplicate data related to the file that is held by the own storage node in the cache data storage file.
13. The distributed storage device according to claim 11 , wherein
a predetermined plurality of files are access units from the server, the predetermined plurality of files in the access unit are distributed and stored in a plurality of storage nodes, and a distributed storage node that stores the files is determined by the storage node, and
when discarding the cache data, if the cache data to be discarded is a file that configures a part of an access unit, the processor discards the cache data of another file that configures the same access unit as the file.
14. A data management method in a distributed storage device that includes a plurality of storage nodes, wherein
the storage node includes a storage device and a processor,
the plurality of storage nodes have a deduplication function for deduplication between storage nodes,
the storage device stores files that are not deduplicated in the plurality of storage nodes, duplicate data storage files in which deduplicated duplicate data is stored, and cache data storage file in which cache data of duplicate data stored in another storage node is stored,
the cache data is discarded when a predetermined condition is satisfied, and
when a read access request for the cache data is received, the cache data is read if the cache data is stored in the cache data storage file, and another storage node is requested to read the duplicate data related to the cache data if the cache data is discarded.