Storage of data reference blocks and deltas in different storage devices
View Patent ↗A data storage architecture is composed of an array of a flash memory solid state disk and a hard disk drive or any nonvolatile random access storage that are intelligently coupled by an intelligent processing unit such as a multi-core graphic processing unit. The solid state disk stores seldom-changed and mostly read reference data blocks while the hard disk drive stores compressed deltas between currently accessed I/O blocks and their corresponding reference blocks in the solid state disk so that random writes are not performed on the solid state disk during online I/O operations. The solid state disk and hard disk drive are controlled by the intelligent processing unit, which carries out high speed computations including similarity detection and delta compression/decompression. The architecture exploits the fast read performance of solid state disks and the high speed computation of graphic processing units to replace mechanical operations on hard disk drives while avoiding slow and wearing solid state drive writes.
1. A system for storing and retrieving data, comprising:
a host computer;
at least a first solid state memory drive;
at least a first random access memory cache;
at least a first hard disk drive;
an intelligent processing unit, wherein the intelligent processing unit is coupled with the host computer, wherein the first solid state memory drive is coupled with the intelligent processing unit, wherein the first random access memory cache is coupled with the intelligent processing unit, wherein the first hard disk drive is coupled with the intelligent processing unit;
wherein the intelligent processing unit is operable to respond to a first read request from the host computer to read a data block within a pre-defined window;
wherein the intelligent processing unit is operable to search for a first reference block stored in the first solid state memory drive corresponding to the data block;
wherein the intelligent processing unit is operable to locate the first reference block corresponding to the data block on the first solid state memory drive;
wherein the intelligent processing unit is operable to locate a first delta corresponding to the data block from at least one of the first random access memory cache and a delta block stored on the first hard disk drive;
wherein the intelligent processing unit is operable to combine the first delta and the first reference block to form the requested data block;
wherein the intelligent processing unit is operable to return the requested data block to the host computer;
wherein the intelligent processing unit is operable to determine whether each of a plurality of additional read requests received within the pre-defined window is addressed to an additional data block having a corresponding second delta that is stored in the same delta block on the hard disk drive as the first delta; and
wherein the intelligent processing unit is operable to combine each of the plurality of additional read requests determined to be addressed to an additional data block having a corresponding second delta that is stored in the same delta block on the hard disk drive as the first delta with the first read request to perform a single hard disk drive operation, wherein the first delta and the second delta are stored in the same delta block based on a temporal locality of access for the first data block and the additional data block.
2. The system of claim 1 , wherein the intelligent processing unit is further operable to locate an independent data block, in at least one of the first solid state memory drive and the first hard disk drive, as the requested data block, when there is no reference block stored in the first solid state memory drive corresponding to the requested data block.
3. The system of claim 2 , wherein the intelligent processing unit is further operable to return the independent data block to the host computer.
4. The system of claim 1 , wherein the intelligent processing unit comprises at least one of a multi-core microprocessor or a graphics processing unit.
5. The system of claim 1 , wherein the delta block is not stored on the first solid state memory drive, and wherein the first reference block is not stored on the first hard disk drive.
6. The system of claim 1 , wherein the intelligent processing unit is operable to advance the pre-defined window forward and identify a plurality of read requests in the advanced window that each has a corresponding delta stored in a same delta block.
7. The system of claim 6 , wherein the intelligent processing unit is operable to combine the identified plurality of read requests to perform a single hard disk drive operation.
8. The system of claim 1 , wherein the intelligent processing unit is further operable to periodically update reference blocks stored in the flash memory.
9. A system for storing and retrieving data, comprising:
a host computer;
a storage controller, the storage controller comprising a host interface, a flash memory, a random access memory cache, a hard disk drive interface, and an intelligent processing unit, wherein the host interface is coupled with the host computer, wherein the intelligent processing unit is coupled with the host interface, wherein the flash memory is coupled with the intelligent processing unit, wherein the random access memory cache is coupled with the intelligent processing unit, wherein the hard disk drive interface is coupled with the intelligent processing unit, wherein the hard disk drive interface is coupled with a hard disk drive;
wherein the intelligent processing unit is operable to respond to a first read request to read from the host computer a data block within a pre-defined window;
wherein the intelligent processing unit is operable to search for a first reference block stored in the flash memory corresponding to the data block;
wherein the intelligent processing unit is operable to locate the first reference block corresponding to the data block in the flash memory;
wherein the intelligent processing unit is operable to locate a first delta corresponding to the data block from at least one of the random access memory cache and a delta block stored on the hard disk drive;
wherein the intelligent processing unit is operable to combine the first delta and the first reference block to form the requested data block;
wherein the intelligent processing unit is operable to return the requested data block to the host computer;
wherein the intelligent processing unit is operable to determine whether each of a plurality of additional read requests received within the pre-defined window is addressed to an additional data block having a corresponding second delta that is stored in the same delta block on the hard disk drive as the first delta; and
wherein the intelligent processing unit is operable to combine each of the plurality of additional read requests determined to be addressed to an additional data block having a corresponding second delta that is stored in the same delta block on the hard disk drive as the first delta with the first read request to perform a single hard disk drive operation, wherein the first delta and the second delta are stored in the same delta block based on a temporal locality of access for the first data block and the additional data block.
10. The system of claim 9 , wherein the intelligent processing unit is further operable to locate an independent data block, in at least one of the flash memory and the hard disk drive, as the requested data block, when there is no reference block stored in the flash memory corresponding to the requested data block.
11. The system of claim 10 , wherein the intelligent processing unit is further operable to return the independent data block to the host computer.
12. The system of claim 9 , wherein the intelligent processing unit is further operable to periodically update reference blocks stored in the flash memory.
13. The system of claim 9 , wherein the intelligent processing unit is at least one of a multi-core microprocessor and a graphic processing unit.
14. The system of claim 9 , wherein the intelligent processing unit is operable to advance the pre-defined window forward and identify a plurality of read requests in the advanced window that each has a corresponding delta stored in a same delta block and to combine the identified plurality of read requests to perform a single hard disk drive operation.
15. A method for storing and retrieving data, comprising:
an intelligent processing unit receiving a first read request from a host computer to read a data block within a pre-defined window;
the intelligent processing unit searching for a first reference block stored in a solid state memory drive corresponding to the data block;
the intelligent processing unit locating the first reference block corresponding to the data block on the solid state memory drive;
the intelligent processing unit locating a first delta corresponding to the data block from at least one of a random access memory cache and a delta block stored on a hard disk drive;
the intelligent processing unit combining the first delta and the first reference block to form the requested data block;
the intelligent processing unit returning the requested data block to the host computer;
the intelligent processing unit determining whether each of a plurality of additional read requests received within the pre-defined window is addressed to an additional data block having a corresponding second delta that is stored in the same delta block on the hard disk drive as the first delta; and
the intelligent processing unit combining each of the plurality of additional read requests determined to be addressed to an additional data block having a corresponding second delta that is stored in the same delta block on the hard disk drive as the first delta with the first read request to perform a single hard disk drive operation, wherein the first delta and the second delta are stored in the same delta block based on a temporal locality of access for the first data block and the additional data block.
16. The method of claim 15 , further comprising: the intelligent processing unit locating an independent data block, in at least one of the solid state memory drive and the hard disk drive, as the requested data block, when there is no reference block stored in the solid state memory drive corresponding to the requested data block.
17. The method of claim 16 , further comprising: the intelligent processing unit returning the independent data block to the host computer.
18. The method of claim 15 , further comprising: the intelligent processing advancing the pre-defined window forward; and identifying a plurality of read requests in the advanced window that each has a corresponding delta stored in a same delta block; and combining the identified plurality of read requests to perform a single hard disk drive operation.
19. The method for storing and retrieving data as claimed in claim 15 , further comprising: the intelligent processing unit updating periodically reference blocks stored in the flash memory.
20. The method of claim 15 , wherein the intelligent processing unit is at least one of a multi-core microprocessor and a graphic processing unit.