Duplicate storage elimination is performed in units of block data generated by dividing a data stream into arbitrary-sized blocks. Further, sub-block data is generated by further dividing the block data into a plurality of pieces of data, and sub-address data based on the data content of each of the pieces of sub-block data is stored in a predetermined storage device. As such, duplicate storage elimination is also performed in sub-block data units based on the sub-address data.
1. A storage system including a processor, comprising:
a block data deduplication unit implemented by the processor that stores, in a storage device, block data generated by dividing a data stream into arbitrary-sized blocks, the block data being pointed to by address data based on a data content of the block data, and when attempting to store, in the storage device, another piece of block data having a data content identical to the data content of the block data having been stored in the storage device, stores, in the storage device, the address data pointing to the block data having been stored in the storage device as the other piece of block data to thereby perform duplicate storage elimination of the block data; and
a sub-block data deduplication unit implemented by the processor that stores, in the storage device, sub-block data generated by further dividing the block data into a plurality of pieces of data, each of the pieces of sub-block data being pointed to by a piece of sub-address data based on a data content of the sub-block data, stores a sub-address data group including pieces of sub address data in a predetermined storage device, and when attempting to store, in the storage device, another piece of sub-block data having a data content identical to the data content of the sub-block data having been stored in the storage device, stores, in the predetermined storage device, the piece of sub-address data pointing to the sub-block data having been stored in the storage device as the other piece of sub-block data to thereby perform duplicate storage elimination of the sub-block data, wherein
if a feature value detected from at least one piece of block data in the data stream based on a data content of the block data satisfies a set condition, the sub-block data deduplication unit stores, in the predetermined storage device, the sub-address data group pointing to pieces of sub-block data generated by dividing another piece of block data corresponding to the block data from which the feature value satisfying the set condition is detected, in association with the feature value satisfying the set condition.
2. The storage system, according to claim 1 , wherein
the sub-block data deduplication unit performs duplicate storage elimination of the sub-block data on the pieces of sub-block data generated by dividing the other block data on which duplicate storage elimination is not performed by the block data deduplication unit.
3. The storage system, according to claim 1 , wherein
if a feature value detected from at least one piece of block data in the data stream based on a data content of the block data satisfies a set condition, the sub-block data deduplication unit stores, in the predetermined storage device, the sub-address data group pointing to pieces of sub-block data generated by dividing another piece of block data adjacent to the block data from which the feature value satisfying the set condition is detected, in association with the feature value satisfying the set condition.
4. The storage system, according to claim 3 , wherein
if the feature value detected from the piece of block data in the data stream has already been stored in the predetermined storage device, the sub-block data deduplication unit compares the sub-address data group having been stored in association with the feature value with a sub-address data group pointing to pieces of sub-block data constituting another block data adjacent to the block data from which the feature value is detected, to thereby perform duplicate storage elimination of the sub-block data.
5. The storage system, according to claim 4 , wherein
if a plurality of pieces of the sub-block data are not stored in the storage device as a result of comparison between the sub-address data groups, the sub-block data deduplication unit stores the pieces of sub-block data by combining them.
6. The storage system, according to claim 3 , wherein
the sub-block data deduplication unit stores the feature value satisfying the set condition in the predetermined storage device in association with a plurality of sub-address data groups.
7. The storage system, according to claim 2 , wherein the sub-block data deduplication unit stores the sub-address data group associated with the feature value satisfying the set condition in the pre-determined storage device in which a reading and writing speed is higher than that of the storage device storing the block data and the sub-block data.
8. A non-transitory computer-readable medium storing a program comprising instructions for causing an information processor device to realize:
a block data deduplication unit that stores, in a storage device, block data generated by dividing a data stream into arbitrary-sized blocks, the block data being pointed to by address data based on a data content of the block data, and when attempting to store, in the storage device, another piece of block data having a data content identical to the data content of the block data having been stored in the storage device, stores, in the storage device, the address data pointing to the block data having been stored in the storage device as the other piece of block data to thereby perform duplicate storage elimination of the block data; and
a sub-block data deduplication unit that stores, in the storage device, sub-block data generated by further dividing the block data into a plurality of pieces of data, each of the pieces of sub-block data being pointed to by a piece of sub-address data based on a data content of the sub-block data, stores a sub-address data group including pieces of sub-address data in a predetermined storage device, and when attempting to store, in the storage device, another piece of sub-block data having a data content identical to the data content of the sub-block data having been stored in the storage device, stores, in the predetermined storage device, the piece of sub-address data pointing to the sub-block data having been stored in the storage device as the other piece of sub-block data to thereby perform duplicate storage elimination of the sub-block data, wherein
if a feature value detected from at least one piece of block data in the data stream based on a data content of the block data satisfies a set condition, the sub-block data deduplication unit stores, in the predetermined storage device, the sub-address data group pointing to pieces of sub-block data generated by dividing another piece of block data corresponding to the block data from which the feature value satisfying the set condition is detected, in association with the feature value satisfying the set condition.
9. The non-transitory computer-readable medium storing the program according to claim 8 , wherein
the sub-block data deduplication unit performs duplicate storage elimination of the sub-block data on the pieces of sub-block data generated by dividing the other block data on which duplicate storage elimination is not performed by the block data deduplication unit.
10. The non-transitory computer-readable medium storing the program according to claim 8 , wherein
if a feature value detected from at least one piece of block data in the data stream based on a data content of the block data satisfies a set condition, the sub-block data deduplication unit stores, in the predetermined storage device, the sub-address data group pointing to pieces of sub-block data generated by dividing another piece of block data adjacent to the block data from which the feature value satisfying the set condition is detected, in association with the feature value satisfying the set condition.
11. An information processing method, comprising:
storing, in a storage device, block data generated by dividing a data stream into arbitrary-sized blocks, the block data being pointed to by address data based on a data content of the block data, and when attempting to store, in the storage device, another piece of block data having a data content identical to the data content of the block data having been stored in the storage device, storing, in the storage device, the address data pointing to the block data having been stored in the storage device as the other piece of block data thereby performing duplicate storage elimination of the block data; and
storing, in the storage device, sub-block data generated by further dividing the block data into a plurality of pieces of data, each of the pieces of sub-block data being pointed to by a piece of sub-address data based on a data content of the sub-block data, storing a sub-address data group including pieces of sub-address data in a predetermined storage device, and when attempting to store, in the storage device, another piece of sub-block data having a data content identical to the data content of the sub-block data having been stored in the storage device, storing, in the predetermined storage device, the piece of sub-address data pointing to the sub-block data having been stored in the storage device as the other piece of sub-block data thereby performing duplicate storage elimination of the sub-block data, wherein
if a feature value detected from at least one piece of block data in the data stream based on a data content of the block data satisfies a set condition, storing, in the predetermined storage device, the sub-address data group pointing to pieces of sub-block data generated by dividing another piece of block data corresponding to the block data from which the feature value satisfying the set condition is detected, in association with the feature value satisfying the set condition.
12. The information processing method, according to claim 11 , further comprising
performing duplicate storage elimination of the sub-block data on the pieces of sub-block data generated by dividing the other block data on which duplicate storage elimination is not performed.
13. The information processing method, according to claim 12 , further comprising
if a feature value detected from at least one piece of block data in the data stream based on a data content of the block data satisfies a set condition, storing, in the predetermined storage device, the sub-address data group pointing to pieces of sub-block data generated by dividing another piece of block data adjacent to the block data from which the feature value satisfying the set condition is detected, in association with the feature value satisfying the set condition.