File system consistency in a distributed system using version vectors

1. A method, comprising:

obtaining, by a first backup node from a storage repository, a current version vector associated with a file and a last completed version vector associated with the file, wherein the file is accessible by a distributed file system with a plurality of backup nodes that at least includes the first backup node and a second backup node, wherein the current version vector associated with the file is representative of a current operation being performed on the file, wherein the current version vector associated with the file is comprised of a backup node identifier associated with a backup node performing the current operation on the file, a first incarnation identifier associated with the backup node performing the current operations on the file, and a first transaction identifier, wherein the last completed version vector is representative of a most recent successfully completed operation performed on the file, wherein the last completed version vector associated with the file is comprised of a second node identifier associated with a backup node that performed the most recent successfully completed operation on the file, a second incarnation identifier associated with the backup node that performed the most recent successfully completed operation on the file, and a second transaction identifier, wherein the first incarnation identifier associated with the backup node performing the current operation on the file indicates a number of times the backup node performing the current operation on the file has been restarted and a first transaction identifier indicates a number of transactions performed by the backup node performing the current operation on the file while the backup node performing the current operation on the file has the first incarnation identifier;

determining, by the first backup node, whether the second backup node is currently performing operations on the file at least in part by:

comparing the first incarnation identifier associated with the backup node performing the current operations on the file of the current version vector of the file with the second incarnation identifier associated with the backup node that performed the most recent successfully completed operation on the file of the last completed version vector associated with the file; and

comparing the first transaction identifier associated with the backup node performing the current operations on the file with the second transaction identifier associated with the backup node that performed the most recent successfully completed operation on the file of the last completed version vector associated with the file;

in response to determining that the second backup node is currently performing operations on the file, waiting, by the first backup node, to access the file until the current version vector associated with the file is equal to the last completed version vector associated with the file; and

in response to the current version vector associated with the file comprising the first incarnation identifier and the first transaction identifier being equal to the last completed version vector associated with the file comprising the second incarnation identifier and the second transaction identifier, accessing, by the backup node, the file at least in part by:

extracting the second incarnation identifier associated with the backup node that performed the most recent successfully completed operation on the file from the last completed version vector associated with the file;

generating an updated version vector prior to reading the file by incrementing the second incarnation identifier of the last completed version vector for subsequent file operations to be performed by the first backup node;

committing to the storage repository the updated version vector to be the current version vector associated with the file;

reading the file;

determining that an inconsistency for the file exists;

fixing the inconsistency in the file; and

committing to the storage repository the updated version vector to be the last completed version vector associated with the file.

2. The method of claim 1 , wherein fixing the inconsistency in the file comprises rolling back the file to a previous stable version of the file.

3. The method of claim 1 , wherein the current version vector associated with the file is stored in a first data repository of the storage repository and the last completed version vector associated with the file is stored in a second data repository of the storage repository.

4. The method of claim 1 , wherein fixing the inconsistency in the file comprises attempting to update the file based on a cached copy of a file update.

5. The method of claim 1 , further comprising:

restarting the first backup node;

obtaining a corresponding incarnation identifier from a previous version vector associated with the first backup node; and

resetting the corresponding incarnation identifier and a corresponding transaction identifier, wherein resetting includes incrementing the corresponding incarnation identifier relative to a previous incarnation identifier and setting the corresponding transaction identifier to an initial, sequential value.

6. The method of claim 1 , wherein the current version vector associated with the file and the last completed version vector associated with the file further comprises a unique identifier of the backup node that performed the most recent successfully completed operation on the file.

7. A computer program product, the computer program product being embodied in a non-transitory computer readable medium and comprising instructions for:

obtaining, by a first backup node from a storage repository, a current version vector associated with a file and a last completed version vector associated with the file, wherein the file is accessible by a distributed file system with a plurality of backup nodes that at least includes the first backup node and a second backup node, wherein the current version vector associated with the file is representative of a current operation being performed on the file, wherein the current version vector associated with the file is comprised of a backup node identifier associated with a backup node performing the current operation on the file, a first incarnation identifier associated with the backup node performing the current operation on the file, and a first transaction identifier, wherein the last completed version vector is representative of a most recent successfully completed operation performed on the file, wherein the last completed version vector associated with the file is comprised of a second node identifier associated with a backup node that performed the most recent successfully completed operation on the file, a second incarnation identifier associated with the backup node that performed the most recent successfully completed operation, and a second transaction identifier, wherein the first incarnation identifier associated with the backup node performing the current operation on the file indicates a number of times the backup node performing the current operation on the file has been restarted and the first transaction identifier indicates a number of transactions performed by the backup node performing the current operation on the file while the backup node performing the current operation has the first incarnation identifier;

determining, by the first backup node, whether the second backup node is currently performing operations on the file at least in part by:

comparing the first incarnation identifier associated with the backup node performing the current operations on the file of the current version vector of the file with the second incarnation identifier associated with the backup node that performed the most recent successfully completed operation on the file of the last completed version vector associated with the file; and

comparing the first transaction identifier associated with the backup node performing the current operations on the file with the second transaction identifier associated with the backup node that performed the most recent successfully completed operation on the file of the last completed version vector associated with the file;

in response to determining that that the second backup node is currently performing operations on the file, waiting, by the first backup node, to access the file until the current version vector associated with the file is equal to the last completed version vector associated with the file; and

in response to the current version vector associated with the file comprising the first incarnation identifier and the first transaction identifier being equal to the last completed version vector associated with the file comprising the second incarnation identifier and the second transaction identifier, accessing, by the first backup node the file at least in part by:

extracting the second incarnation identifier associated with the backup node that performed the most recent successfully completed operation on the file from the last completed version vector associated with the file;

generating an updated version vector prior to reading the file by incrementing the second incarnation identifier of the last completed version vector for subsequent file operations to be performed by the first backup node;

committing to the storage repository the updated version vector to be the current version vector associated with the file;

reading the file;

determining that an inconsistency for the file exists;

fixing the inconsistency in the file; and

committing to the storage repository the updated version vector to be the last completed version vector associated with the file.

8. The computer program product of claim 7 , wherein fixing the inconsistency in the file comprises rolling back the file to a previous stable version of the file.

9. The computer program product of claim 7 , wherein fixing the inconsistency in the file comprises attempting to update the file based on a cached copy of a file update.

10. The computer program product of claim 7 , further comprising instructions for:

restarting the first backup node;

obtaining a corresponding incarnation identifier from a previous version vector associated with the first backup node; and

resetting the corresponding incarnation identifier and a corresponding transaction identifier, wherein resetting includes incrementing the corresponding incarnation identifier relative to a previous incarnation identifier and setting the corresponding transaction identifier to an initial, sequential value.

11. The computer program product of claim 7 , wherein the current version vector associated with the file and the last completed version vector associated with the file further comprises a unique identifier of the backup node that performed the most recent successfully completed operation on the file.

12. A system comprising:

a processor of a first backup node configured to:

obtain, by the first backup node from a storage repository, a current version vector associated with a file a last completed version vector associated with file, wherein the file is accessible by a distributed file system with a plurality of backup nodes that at least includes the first backup node and a second backup node, wherein the current version vector associated with the file is representative of a current operation being performed on the file, wherein the current version vector associated with the file is comprised of a backup node identifier associated with a backup node performing the current operation on the file, a first incarnation identifier associated with the backup node performing the current operations on the file, and a first transaction identifier, wherein the last completed version vector is representative of a most recent successfully completed operation performed on the file, wherein the last completed version vector associated with the file is comprised of a second node identifier associated with a backup node that performed the most recent successfully completed operation on the file, a second incarnation identifier associated with the backup node that performed the most recent successfully completed operation on the file, and a second transaction identifier, wherein the first incarnation identifier associated with the backup node performing the current operation on the file indicates a number of times a backup node performing the current operation on the file been restarted and a transaction identifier indicates a number of transactions performed by the backup node performing the current operation on the file while the backup node performing the current operation on the file has the first incarnation identifier;

determine, by the first backup node, whether the second backup node is currently performing operations on the file at least in part by:

comparing the first incarnation identifier associated with the backup node performing the current operations on the file of the current version vector of the file with the second incarnation identifier associated with the backup node that performed the most recent successfully completed operation on the file of the last completed version vector associated with the file; and

comparing the first transaction identifier associated with the backup node performing the current operations on the file with the second transaction identifier associated with the backup node that performed the most recent successfully completed operation on the file of the last completed version vector associated with the file;

in response to determining that the second backup node is currently performing operations on the file, wait, by the first backup node, to access the file until the current version vector associated with the file is equal to the last completed version vector associated with the file; and

in response to the current version vector associated with the file comprising the first incarnation identifier and the first transaction identifier being equal to the last completed version vector associated with the file comprising the second incarnation identifier and the second transaction identifier, the processor of the first backup node configured to access the file at least in part by:

extracting the second incarnation identifier associated with the backup node that performed the most recent successfully completed operation on the file from the last completed version vector associated with the file;

generating an updated version vector prior to reading the file by incrementing the second incarnation identifier of the last completed version vector for subsequent file operations to be performed by the first backup node;

committing to the storage repository the updated version vector to be the current version vector associated with the file;

reading the file;

determining that an inconsistency for the file exists;

fixing the inconsistency in the file; and

committing to the storage repository the updated version vector to be the last completed version vector associated with the file; and

a memory coupled to the processor and configured to provide the processor with instructions.

13. The system of claim 12 , wherein to fix the inconsistency, the processor is configured to rolling back the file to a previous stable version of the file.

14. The system of claim 12 , wherein to fix the inconsistency, the processor is configured to attempt to update the file based on a cached copy of a file update.

15. The system of claim 12 , wherein the processor is further configured to:

restart the first backup node;

obtain a corresponding incarnation identifier from a previous version vector associated with the first backup node; and

reset the corresponding incarnation identifier and a corresponding transaction identifier, wherein to reset the first incarnation identifier and the corresponding transaction identifier, the processor is further configured to increment the first incarnation identifier relative to a previous incarnation identifier and setting the corresponding transaction identifier to an initial, sequential value.

16. The system of claim 12 , wherein the current version vector associated with the file and the last completed version vector associated with the file further comprises a unique identifier of the backup node that performed the most recent successfully completed operation on the file.