Efficient data backup in a distributed storage system
A method and a server system for creating consistent snapshots in a distributed storage system are disclosed. The server system stores entities in databases that includes a production portion for storing current versions of entities and a backup portion for storing backup versions of modified entities, receives a request for a snapshot of a portion of the databases at a predefined snapshot time. In response to the request, the server system generates a snapshot of the databases, the snapshot including: a backup version of an entity retrieved from the backup portion and a current version of another entity retrieved from the databases and produces a response to the request based on the snapshot. When there are several backup portions, a backup portion may be used separately or in conjunction with other backup portions to provide required backup versions to generate a snapshot.
1. A method comprising:
receiving, by one or more processors, a request for a snapshot of data stored in one or more databases, the request having a snapshot time;
determining, by the one or more processors, that an amount of time that has elapsed since the snapshot time is less than a safe age;
delaying, by the one or more processors, generation of the snapshot until the amount of time that has elapsed since the snapshot time is greater than the safe age; and
generating, by the one or more processors, the snapshot.
2. The method of claim 1 , further comprising storing, by the one or more processors, a plurality of entities in a set of one or more databases, the set of one or more databases including a production portion for storing current versions of entities and a backup portion for storing backup versions of modified entities.
3. The method of claim 2 , further comprising:
receiving, by the one or more processors, a request to modify a respective entity, wherein the respective entity was last modified prior to the snapshot time; and
in response to the request to modify the respective entity:
storing, by the one or more processors, a backup version of the respective entity in the backup portion of the set of one or more databases; and
modifying, by the one or more processors, the respective entity in the production portion of the set of one or more databases.
4. The method of claim 2 , wherein the snapshot comprises data from the backup portion but not the production portion for entities last modified after the snapshot time, and data from the production portion but not the backup portion for entities last modified before the snapshot time.
5. The method of claim 2 , further comprising:
receiving, by the one or more processors, a request to modify a second entity that was last modified prior to the snapshot time; and
in response to the request to modify the second entity:
storing, by the one or more processors, a backup version of the second entity in the backup portion of the set of one or more databases; and
modifying, by the one or more processors, the second entity in the production portion of the set of one or more databases.
6. The method of claim 5 , wherein:
a backup version of a first entity is stored prior to processing the request for the snapshot; and
the snapshot comprises the backup version of the second entity that is modified while processing the request for the snapshot.
7. The method of claim 1 , further comprising receiving, by the one or more processors, a snapshot timestamp corresponding to the snapshot time.
8. The method of claim 1 , further comprising:
receiving, by the one or more processors, a request to store the snapshot; and
storing, by the one or more processors, the snapshot.
9. A system comprising one or more processors configured to:
receive a request for a snapshot of data stored in one or more databases, the request having a snapshot time;
determine that an amount of time that has elapsed since the snapshot time is less than a safe age;
delay generation of the snapshot until the amount of time that has elapsed since the snapshot time is greater than the safe age; and
generate the snapshot.
10. The system of claim 9 , wherein the one or more processors are further configured to store a plurality of entities in a set of one or more databases, the set of one or more databases including a production portion for storing current versions of entities and a backup portion for storing backup versions of modified entities.
11. The system of claim 10 , wherein the one or more processors are further configured to:
receive a request to modify a respective entity, wherein the respective entity was last modified prior to the snapshot time; and
in response to the request to modify the respective entity:
store a backup version of the respective entity in the backup portion of the set of one or more databases; and
modify the respective entity in the production portion of the set of one or more databases.
12. The system of claim 10 , wherein the snapshot comprises data from the backup portion but not the production portion for entities last modified after the snapshot time, and data from the production portion but not the backup portion for entities last modified before the snapshot time.
13. The system of claim 10 , wherein the one or more processors are further configured to:
receive a request to modify a second entity that was last modified prior to the snapshot time; and
in response to the request to modify the second entity:
store a backup version of the second entity in the backup portion of the set of one or more databases; and
modify the second entity in the production portion of the set of one or more databases.
14. The system of claim 13 , wherein:
a backup version of a first entity is stored prior to processing the request for the snapshot; and
the snapshot comprises the backup version of the second entity that is modified while processing the request for the snapshot.
15. The system of claim 9 , wherein the one or more processors are further configured to receive a snapshot timestamp corresponding to the snapshot time.
16. The system of claim 9 , wherein the one or more processors are further configured to:
receive a request to store the snapshot; and
store the snapshot.
17. One or more non-transitory computer readable storage media storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving a request for a snapshot of data stored in one or more databases, the request having a snapshot time;
determining that an amount of time that has elapsed since the snapshot time is less than a safe age;
delaying generation of the snapshot until the amount of time that has elapsed since the snapshot time is greater than the safe age; and
generating the snapshot.
18. The one or more non-transitory computer readable storage media of claim 17 , wherein the operations further comprise storing a plurality of entities in a set of one or more databases, the set of one or more databases including a production portion for storing current versions of entities and a backup portion for storing backup versions of modified entities.
19. The one or more non-transitory computer readable storage media of claim 17 , wherein the operations further comprise receiving a snapshot timestamp corresponding to the snapshot time.
20. The one or more non-transitory computer readable storage media of claim 17 , wherein the operations further comprise:
receiving a request to store the snapshot; and
storing the snapshot.