DE-DUPLICATING MULTI-DEVICE PLUGIN
Systems, methods, and devices are disclosed herein for implementing deduplicating multi-device plugin. Methods may include receiving a data storage request identifying a data block for storage in a virtual device, where the virtual device is created by a multiple device driver capable of generating a plurality of virtual devices based on a plurality of underlying physical storage devices and a plurality of remote devices. The methods may also include determining, using one or more processors, whether the data block has already been stored in the virtual device created by the multiple device driver. The methods may further include updating, using the one or more processors, a blockmap based on the determining, the blockmap representing a plurality of data blocks stored in the virtual device.
1 . A method comprising:
receiving a data storage request identifying a data block for storage in a virtual device, the virtual device being created by a multiple device driver capable of generating a plurality of virtual devices based on a plurality of underlying physical storage devices and a plurality of remote devices;
determining, using one or more processors, whether the data block has already been stored in the virtual device created by the multiple device driver; and
updating, using the one or more processors, a blockmap based on the determining, the blockmap representing a plurality of data blocks stored in the virtual device.
2 . The method of claim 1 , wherein the virtual device is accessible by a local machine in which the multiple device driver is installed.
3 . The method of claim 2 , wherein the creating of the virtual device comprises:
generating a remote block device container associated with the virtual device;
generating a block device unit within the block device container; and
automatically populating a blockmap associated with the block device unit within the block device container.
4 . The method of claim 1 , wherein the determining whether the data block has already been stored in the virtual device further comprises:
generating a representation of the identified data block by fingerprinting the identified data block;
looking up the representation of the identified data block in an index of fingerprints of stored data blocks; and
determining whether or not the representation of the identified data block exists in a deduplication repository.
5 . The method of claim 4 , wherein the determining whether the data block has already been stored in the virtual device uses a remote protocol.
6 . The method of claim 4 , wherein the updating of the index comprises:
updating a data block reference count associated with the virtual device.
7 . The method of claim 6 further comprising:
providing the identified data block to a networked storage device.
8 . The method of claim 7 , wherein the networked storage device is a deduplication repository.
9 . The method of claim 1 , wherein the multiple device driver is a Linux-compatible driver.
10 . The method of claim 9 , wherein the multiple device driver is implemented on a Linux-based local machine.
11 . A device comprising:
a communications interface configured to be communicatively coupled with a networked storage device; and
one or more processors configured to:
receive a data storage request identifying a data block for storage in a virtual device, the virtual device being created by a multiple device driver capable of generating a plurality of virtual devices based on a plurality of underlying physical storage devices and a plurality of remote devices;
determine whether the data block has already been stored in the virtual device created by the multiple device driver; and
update a blockmap based on the determining, the blockmap representing a plurality of data blocks stored in the virtual device.
12 . The device of claim 11 , wherein the virtual device is accessible by a local machine in which the multiple device driver is installed, and wherein the networked storage device is configured to:
generate a remote block device container associated with the virtual device;
generate a block device unit within the block device container; and
automatically populate a blockmap associated with the block device unit within the block device container.
13 . The device of claim 11 , wherein the one or more processors are further configured to:
generate a representation of the identified data block by fingerprinting the identified data block;
look up the representation of the identified data block in an index of fingerprints of stored data blocks; and
determine whether or not the representation of the identified data block exists in a deduplication repository.
14 . The device of claim 13 , wherein the one or more processors are configured to determine whether the data block has already been stored in the virtual device using a remote protocol.
15 . The device of claim 13 , wherein the one or more processors are further configured to:
update a data block reference count associated with the virtual device; and
provide the identified data block to a networked storage device.
16 . A system comprising:
a networked storage device; and
a local machine comprising one or more processors configured to:
receive a data storage request identifying a data block for storage in a virtual device, the virtual device being created by a multiple device driver capable of generating a plurality of virtual devices based on a plurality of underlying physical storage devices and a plurality of remote devices;
determine whether the data block has already been stored in the virtual device created by the multiple device driver; and
update a blockmap based on the determining, the blockmap representing a plurality of data blocks stored in the virtual device.
17 . The system of claim 16 , wherein the virtual device is accessible by a local machine in which the multiple device driver is installed, and wherein the networked storage device is configured to:
generate a remote block device container associated with the virtual device;
generate a block device unit within the block device container; and
automatically populate a blockmap associated with the block device unit within the block device container.
18 . The system of claim 16 , wherein the one or more processors are further configured to:
generate a representation of the identified data block by fingerprinting the identified data block;
look up the representation of the identified data block in an index of fingerprints of stored data blocks; and
determine whether or not the representation of the identified data block exists in a deduplication repository.
19 . The system of claim 18 , wherein the one or more processors are configured to determine whether the data block has already been stored in the virtual device using a remote protocol.
20 . The system of claim 18 , wherein the one or more processors are further configured to:
update a data block reference count associated with the virtual device; and
provide the identified data block and the updated blockmap to a networked storage device.