Fast recovery of backup cloud gateway following crash without garbage collection

1. A method of recovering from a crash of a system that stores backup images of data objects in backup storage without the necessity of garbage collection, comprising:

creating a plurality of chunk objects from a data object to be backed up, for a each chunk object comprising a different portion of said data object;

generating for each chunk object a unique fingerprint;

assigning a generation number to said data object, said generation number comprising a unique non-repeating number that identifies a time of storage of said data object in backup storage relative to other data objects;

prefixing each of said plurality of chunk objects created from said data object with said generation number assigned to said data object from which each of said chunk objects was created and with said unique fingerprint of each said chunk object to identify each of said plurality of chunk objects and to associate each of said chunk objects with said data object from which each chunk was created;

listing in a manifest file for said data object each of said prefixed chunk objects in an order of appearance of said chunk object in said data object, and assigning to said manifest file as an identifier the same generation number as assigned to said data object;

storing said prefixed chunk objects in backup storage and storing said manifest file in persistent storage;

upon a crash of said system occurring, deleting all chunk objects from said backup storage that are prefixed by a generation number that is later than a last generation number that was persistently stored prior to said crash; and

recreating using said manifest file and unique fingerprint and storing in said backup storage prefixed with a new generation number replacement chunk objects corresponding to said deleted chunk objects to return said backup storage to a consistent state.

2. The method of claim 1 , wherein said generation number comprises a forever increasing number that is consecutively incremented for each new data object.

3. The method of claim 1 , wherein said generation number comprises a number of a sequence that has a value that distinguishes its relative rank in the sequence to other numbers of the sequence.

4. The method of claim 1 further comprising associating each chunk object with an offset and a length of said each chunk object relative to a reference location in said data object.

5. The method of claim 4 further comprising upon a crash deleting said manifest file corresponding to said data object.

6. The method of claim 1 , wherein said backup storage comprises cloud storage, and said creating, said generating and said prefixing are performed by a cloud gateway communicating with said data processing system, and wherein said manifest file is additionally stored in a cloud gateway interfaced to said data processing system.

7. The method of claim 6 , wherein said cloud gateway generates said generation number, and stores said generation number and said manifest file in persistent storage in said cloud gateway.

8. The method of claim 6 , wherein said cloud storage comprises persistent storage, and said manifest file and said generation number are stored in persistent storage.

9. A method of recovering from a crash of a system that stores backup images of data objects in cloud storage without the necessity of garbage collection, comprising:

breaking up a data object into a plurality of chunk objects, each chunk object comprising a different portion of said data object;

creating for and associating with each chunk object a fingerprint that uniquely identifies said chunk object;

assigning a generation number to said data object and prefixing the same generation number to each of said plurality of chunk objects created from said data object, said generation number comprising a unique number that identifies a time of storage of said data object in backup storage relative to other data objects, and that associates each of said plurality of chunk objects with said data object;

listing each of said prefixed chunk objects in a manifest file in order of their appearance in said data object along with metadata indicating a location of each chunk object in said data object;

storing said plurality of chunk objects prefixed with said generation number in said cloud storage;

upon a crash of said system occurring, deleting all chunk objects in cloud storage having with a generation number that is greater than a last consistent generation number that was associated with a consistent state of the cloud storage prior to the crash; and

recreating in cloud storage using said manifest file and said fingerprint replacement chunk objects for said chunk objects that were deleted to return said cloud storage to a consistent state.

10. The method of claim 9 , wherein said generation number is consecutively incremented.

11. The method of claim 9 further comprising, identifying said manifest file by said generation number.

12. The method of claim 9 further comprising storing said manifest file and said generation number in persistent storage in said cloud storage and in a cloud gateway interfaced to said cloud storage.

13. Non-transitory computer readable media storing executable instructions for controlling the operations of a computer to recover from a crash of a system that stores backup images of data objects in cloud storage without the necessity of garbage collection, comprising:

creating a plurality of chunk objects from a data object, each chunk object comprising a different portion of said data object;

generating for each data chunk a unique fingerprint;

assigning a generation number to said data object, said generation number comprising a unique non-repeating number that identifies a time of backup of said data object in backup storage relative to other data objects;

prefixing each of said plurality of chunk objects created from said data object with said generation number assigned to said data object from which each of said chunk objects was created and with said unique fingerprint of each said chunk object to identify each of said plurality of chunk objects as having been created from said data object;

listing each of said prefixed chunk objects in a manifest file in order of their appearance in said data object, and assigning to said manifest file as an identifier the same generation number as assigned to said data object;

upon a crash of said system occurring, deleting all chunk objects from said cloud storage that are prefixed by a generation number that is later than a last generation number that was persistently stored prior to said crash; and

recreating using said manifest file and said unique fingerprint and storing new replacement chunk objects for said chunk objects of said data object that were deleted to return said storage to a consistent state.

14. The non-transitory computer readable media of claim 13 , wherein said generation number comprises a forever increasing number that is consecutively incremented for each new data object.

15. The non-transitory computer readable media of claim 13 further comprising associating each chunk object with an offset and a length of each chunk object relative to a reference location in said data object.

16. The non-transitory computer readable media of claim 15 further comprising upon a crash deleting said manifest file corresponding to said data object.

17. The non-transitory computer readable media of claim 13 , further comprising a cloud gateway that interfaces with said cloud storage, and storing said manifest file and said generation number in persistent storage in said cloud gateway and in said cloud storage.