System and method for duplicating electronically stored data
View Patent ↗A method of duplicating electronically stored data that includes coupling a duplicating device between an original information storage device and a destination storage device, and streaming data from the original information storage device to the duplicating device. An embodiment of the method also includes segmenting the streaming data, determining a length of an uncompressed data segment, and calculating a length of the data segment when compressed. This embodiment further includes determining if the length of the data segment when compressed is less than some threshold, and copying a compressed data segment to the destination storage device if the length of the compressed data segment is less than the threshold, otherwise copying the uncompressed data segment to the destination storage device.
1. A method for duplicating electronically stored data comprising:
coupling a duplicating device between a source information storage device and a destination storage device;
streaming data from the source information storage device to the duplicating device;
segmenting the streaming data;
determining a length of an uncompressed data segment;
calculating a length of the corresponding data segment when compressed;
determining if the length of the compressed data segment is less than a threshold; and
copying the compressed data segment to an output file register if the length of the compressed data segment is less than the threshold, otherwise copying the uncompressed data segment to the output file register.
2. The method of claim 1 , further comprising creating a directory that includes information on each data segment, wherein the information includes an absolute byte address for the corresponding data segment in the output file, and wherein the information is compacted and stored in a memory device.
3. The method of claim 2 , further comprising copying data from the directory to the output file register.
4. The method of claim 3 , wherein copying data from the directory to the output file register comprises formatting the directory data before writing it to the output file register.
5. The method of claim 4 , wherein formatting the directory data comprises decompacting the directory data for a particular data segment by substituting an absolute byte address for a byte offset from a base address in the directory information, wherein the byte offset is determined by the lengths of data segments prior to the particular data segment.
6. The method of claim 1 , wherein segmenting the streaming data comprises segmenting the streaming data into segments of a predetermined length.
7. The method of claim of claim 6 , wherein segmenting the streaming data into segments of a predetermined length comprises segmenting the streaming data into segments of approximately 32 kilobytes.
8. The method of claim 1 , wherein calculating the length of a compressed segment comprises applying a subset of an LZ77-based compression algorithm to the data stream, then counting the number of bytes in the data stream after application of the compression algorithm.
9. The method of claim 1 , wherein determining if the length of the data segment when compressed is less than the threshold comprises determining if the length of the data segment when compressed is less than a threshold that is less than the length of the uncompressed segment.
10. The method of claim 1 , wherein streaming data from the source information storage device comprises streaming data from the source information storage device into a FIFO register.
11. The method of claim 1 , further comprising writing contents of the output file register to the destination storage device.
12. The method of claim 11 , further comprising calculating a checksum for each data segment copied to the destination storage device.
13. The method of claim 12 , wherein calculating a checksum comprises calculating an Adler-32 checksum.
14. The method of claim 1 , wherein copying compressed and uncompressed data segments to an output file register comprises copying compressed and uncompressed data segments to a FIFO register.
15. A system for duplicating electronically stored data, the system comprising:
an input device configured to receive a data stream from a source information storage device;
a byte counting device configured to determine a length of a segment of the data stream, when the segment is uncompressed;
a compression look-ahead device configured to calculate the length of the segment of the data stream, when the segment is compressed; and
a data output device configured to write the uncompressed segment of the data stream and the compressed segment of the data stream to a destination storage medium.
16. The system of claim 15 , wherein the compression look-ahead device comprises:
a run-length encoder;
a Huffman coder coupled to the run-length encoder; and
a byte counter coupled to the Huffman coder.
17. The system of claim 15 , further comprising a bulk FIFO register to store portions of the segment of the data stream.
18. The system of claim 15 , further comprising a compression look-ahead result register coupled to the compression look-ahead unit, and configured to store a result of a comparison between the length of the uncompressed data segment and the length of the compressed data segment.
19. The system of claim 15 , further comprising a checksum calculator configured to add a checksum to a segment of the uncompressed data stream and to a segment of the compressed data stream.
20. The system of claim 19 , wherein the checksum calculator comprises an Adler-32 checksum calculator.
21. The system of claim 15 , wherein the data output device comprises a FIFO register.
22. The system of claim 15 , further comprising a directory register configured to store data for each segment of the data stream in a directory table.
23. The system of claim 22 , further comprising a directory formatter configured to format data from the directory register for storage in the data output device.
24. The system of claim 23 , wherein the directory formatter is further configured to compact data from the directory register before storing the data in a storage device configured for temporary data storage.
25. The system of claim 15 , further comprising a data formatter configured to format data from the bulk FIFO register and from the compression look-ahead device for storage in the data output device.
26. The system of claim 15 , further comprising an output byte counter configured to count the total number of bytes in the data stream written to the destination storage device.
27. A non-transitory machine-readable storage medium having program instructions thereon which when executed by a processor cause the processor to perform the step of:
operating an electronic data duplicating system, wherein operating the electronic data duplicating system comprises:
partitioning streaming data from a source information storage device to a destination storage device into discrete data segments;
determining a length of an uncompressed data segment;
calculating a length of the corresponding data segment when compressed;
determining if the length of the compressed data segment is less than a threshold; and
copying the compressed data segment to an output file register if the length of the compressed data segment is less than the threshold, otherwise copying the uncompressed data segment to the output file register.
28. The non-transitory machine-readable storage medium of claim 27 , wherein operating an electronic data duplicating system further comprises creating a directory that includes information on each data segment, wherein the information includes an absolute byte address for the corresponding data segment in the output file.
29. The non-transitory machine-readable storage medium of claim 28 , wherein operating an electronic data duplicating system further comprises copying data from the directory to the output file register.
30. The non-transitory machine-readable storage medium of claim 29 , wherein copying data from the directory to the output file register comprises formatting the directory data before writing it to the output file register.
31. The non-transitory machine-readable storage medium of claim 30 , wherein formatting the directory data comprises de-compacting the directory data for a particular data segment by substituting an absolute byte address for a byte offset from a base address in the directory information, wherein the byte offset is determined by the lengths of data segments prior to the particular data segment.
32. The non-transitory machine-readable storage medium of claim 27 , wherein operating an electronic data duplicating system further comprises writing the contents of the output file register to the destination storage device.
33. The non-transitory machine-readable storage medium of claim 32 , wherein operating an electronic data duplicating system further comprises calculating a checksum for each data segment copied to the destination storage device.