Method for stream based compressed file download with interruption recovery and further decompressing and de-archiving data in filesystem
View Patent ↗A system and method are provided for storing and using recovery state information during a data stream transfer, such as a download. During the download of compressed, archived data, the system tracks the position of the last file boundary and the position of the last compression block boundary before the last file boundary, and the system stores this information as a recovery state. If the download is interrupted, the system uses the recovery state information to resume the download at an efficient location in the data stream.
1. A method of resuming an interrupted data stream transfer comprising:
obtaining recovery state information, including a stored compression block boundary position and a stored file boundary position of the interrupted data stream transfer;
resuming the data stream transfer by requesting a compressed data stream starting with the stored compression block boundary position;
advancing through the resumed data stream transfer to reach the stored file boundary position by decompressing data from the stored compression block boundary position to the stored file boundary position;
once the file boundary position has been reached, decompressing and de-archiving data after the file boundary position; and
storing the de-archived data in a destination filesystem.
2. The method of claim 1 , wherein requesting a compressed data stream includes sending a range retrieval request.
3. The method of claim 2 , wherein the compression block boundary position includes an offset value from the beginning of the compressed data stream; and
wherein the range retrieval request includes the offset value.
4. The method of claim 1 , further comprising discarding the data decompressed in the first decompressing step.
5. A method of resuming a data stream transfer comprising:
obtaining recovery state information by:
receiving an initial compressed data stream, wherein the initial compressed data stream includes one or more compression block boundaries and one or more file boundaries;
detecting a file boundary; and
in response to detecting the file boundary, storing the position of the detected file boundary and the position of the last compression block boundary before the detected file boundary;
requesting a compressed data stream starting with the stored compression block boundary position;
advancing through the resumed data stream transfer to reach the stored file boundary position by decompressing data from the stored compression block boundary position to the file boundary position; and
once the file boundary position has been reached, decompressing and de-archiving data after the stored file boundary position.
6. The method of claim 5 , wherein requesting a compressed data stream includes sending a range retrieval request.
7. The method of claim 6 , wherein the compression block boundary position includes an offset value from the beginning of the compressed data stream; and
wherein the range retrieval request includes the offset value.
8. The method of claim 5 , further comprising discarding the data decompressed in the first decompressing step.
9. The method of claim 5 , further comprising storing the de-archived data in a destination filesystem.
10. A method of resuming a data stream transfer comprising:
obtaining recovery state information by:
receiving an initial compressed data stream;
detecting a compression block boundary in the initial compressed data stream;
detecting an archive block boundary in the initial compressed data stream;
detecting a file boundary in the initial compressed data stream; and
in response to the detection of the compression block boundary, the archive block boundary, and the file boundary, saving the recovery state information;
wherein the recovery state includes the compression block boundary position and the file boundary position;
requesting a compressed data stream starting with the saved compression block boundary position;
advancing through the resumed data stream transfer to reach the saved file boundary position by decompressing data from the saved compression block boundary position to the saved file boundary position; and
once the file boundary position has been reached, decompressing and de-archiving data after the saved file boundary position.
11. The method of claim 10 , wherein requesting a compressed data stream includes sending a range retrieval request.
12. The method of claim 11 , wherein the compression block boundary position includes an offset value from the beginning of the compressed data stream; and
wherein the range retrieval request includes the offset value.
13. The method of claim 10 , further comprising discarding the data decompressed in the first decompressing step.
14. The method of claim 10 , further comprising storing the de-archived data in a destination filesystem.