Storage system for eliminating duplicated data
A storage system 103 carries out first and second de-duplication processes in response to receiving a write request from a client. First, a determination is made as to whether a write target data item overlaps with any of the stored data items of a part of a stored data item group, which is a user data item group stored in a storage device 209 , and if so, the write target data item is prevented from being stored in the storage device. Second, a determination is made as to whether a target stored data item, which is not finished being evaluated as to whether it overlaps with the stored data item in the first de-duplication process, overlaps with another stored data item, and if so, the target stored data item or the same data item overlapping with the target stored data item is deleted from the storage device 209.
1. A storage system comprising:
a communication interface device coupled to a communication network to which a client is coupled;
a storage device in which a plurality of user data is stored; and
a controller, which is configured to manage ranks of the plurality of user data and select comparison target data in the plurality of user data based on the ranks of the plurality of user data, wherein when the storage system receives a write request from the client, the controller is configured to:
determine whether write target data of the write request overlaps with the selected comparison target data stored in the storage device;
not store the write target data in the storage device and register the write target data as duplicated data when the write target data is determined to overlap with the selected comparison target data; and
store the write target data in the storage device when the write target data is determined not to overlap with the selected comparison target data,
wherein the storage system has a memory,
wherein the selected comparison target data is stored in both the storage device and the memory,
wherein the controller is configured to determine whether the write target data of the write request overlaps with the selected comparison target data stored in the storage device using the selected comparison target data stored in the memory,
wherein a rank of the selected comparison target data is higher than a rank of data which is not selected in the plurality of user data, and
wherein the controller is configured to:
determine whether the write target data conforms with a condition of an asynchronous de-duplication process when the write target data is determined to be duplication unknown data; and
store the write target data in the storage device when the write target data does not conform with the condition of the asynchronous de-duplication process.
2. The storage system according to claim 1 ,
wherein when the write target data conforms with the condition of the asynchronous de-duplication process, the controller is configured to:
store the write target data;
determine whether the stored write target data overlaps with other data in the plurality of user data; and
delete the stored write target data or the other data overlapping with the stored write target data from the storage device as the asynchronous de-duplication process, when the stored write target data is determined to overlap with the other data in the plurality of user data.
3. The storage system according to claim 1 ,
wherein the controller is further configured to:
generate a data hash value of the write target data; and
determine whether the write target data of the write request overlaps with the selected comparison target data stored in the storage device using the generated data hash value of the write target data.