Transactional access to resource repositories
Resources, like source code, can be stored in a repository and managed through a resource repository system. The resource repository system includes a processor and a memory for storing a plurality of data structures, each data structure comprising a current version of a corresponding resource and at least one past version of the corresponding resource. The processor is communicatively coupled to the memory and configured to: request a transaction on a resource of a repository; create a snapshot of the repository; perform reads and/or writes of the transaction against the snapshot; evaluate potential conflicts between concurrently committed transactions; if no conflict is found, commit the transaction by persisting it to a write-ahead log; and update the repository based on the write-ahead log.
1. A resource repository system comprising:
a memory for storing a plurality of data structures, each data structure comprising a current version of a corresponding resource and at least one past version of the corresponding resource; and
a processor communicatively coupled to the memory and configured to:
receive a request for a transaction on a resource of a repository, the transaction making one or more changes to the resource of the repository;
create a snapshot of the repository by:
copying a directory structure of the repository into a temporary directory and hard linking resources of the repository to the temporary directory; or
using a copy-on-write functionality of a filesystem to create a clone of the repository into a temporary directory;
perform reads and/or writes of the transaction on the snapshot;
evaluate potential conflicts between the reads and/or the writes of the transaction and those of other concurrently committed transactions;
responsive to no conflict being found, commit the transaction to a write-ahead log; and
update the repository based on the changes recorded in the write-ahead log.
2. The system of claim 1 , wherein the processor is configured to commit the transaction to the write-ahead log by:
creating a log entry containing the changes to be committed with the transaction; and
persisting the log entry into the write-ahead log.
3. The system of claim 2 , wherein the processor is configured to update the repository by:
reading a transaction's changes from the write-ahead log; and
based on the changes of the transaction, applying the transaction to the repository by performing the recorded changes.
4. The system of claim 1 , wherein the processor is configured to delete the snapshot after committing the transaction to the write-ahead log.
5. The system of claim 1 , wherein evaluating the potential conflicts comprises:
checking transactions logged in the write-ahead log to determine whether the resource is changed by a concurrent transaction that committed concurrently against the repository.
6. The system of claim 1 , wherein the processor is configured to block writes into the repository while one or more snapshots are created to maintain consistency.
7. The system of claim 1 , wherein if a crash occurs while one or more transactions have been committed to the write-ahead log but not yet fully applied to the repository, the processor is configured to recover committed transactions from the write-ahead log and update the repository accordingly.
8. The system of claim 1 , wherein if a conflict is found, the processor is configured to discard the transaction and delete the snapshot associated with the transaction.
9. The system of claim 1 , wherein the processor is configured to provide an alert associated with the conflict if found.
10. A method for performing a transaction on a resource on a database, the method comprising:
receiving a request for a transaction on a resource of a repository, the transaction making one or more changes to the resource of the repository;
creating a snapshot of the repository by:
copying a directory structure of the repository into a temporary directory and hard linking resources of the repository to the temporary directory; or
using a copy-on-write functionality of a filesystem to create a clone of the repository into a temporary directory;
performing reads and/or writes of the transaction on the snapshot;
evaluating potential conflicts between the reads and/or the writes of the transaction and those of other concurrently committed transactions;
responsive to no conflict being found, committing the transaction to a write-ahead log; and
updating the repository based on the changes recorded in the write-ahead log.
11. The method of claim 10 , wherein committing the transaction to the write-ahead log comprises:
creating a log entry containing the changes to be committed with the transaction; and
persisting the log entry into the write-ahead log.
12. The method of claim 11 , wherein updating the repository comprises:
reading a transaction's changes from the write-ahead log; and
based on the changes of the transaction, applying the transaction to the repository by performing the recorded changes.
13. The method of claim 10 , further comprises deleting the snapshot after committing the transaction to the write-ahead log.
14. The method of claim 10 , wherein evaluating the potential conflicts comprises checking transactions logged in the write-ahead log to determine whether the resource is changed by a concurrent transaction that committed concurrently against the repository.
15. The method of claim 10 , further comprising blocking writes into the repository while one or more snapshots are created to maintain consistency.
16. The method of claim 10 , further comprising, if a crash occurs while one or more transactions have been committed to the write-ahead log but not yet fully applied to the repository, recovering committed transactions from the write-ahead log and update the repository accordingly.
17. The method of claim 10 , further comprising, if a conflict is found, discarding the transaction and delete the snapshot associated with the transaction.
18. A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a computing system, cause the computing system to perform operations including:
receiving a request for a transaction on a resource of a repository, the transaction making one or more changes to the resource of the repository;
creating a snapshot of the repository by:
copying a directory structure of the repository into a temporary directory and hard linking resources of the repository to the temporary directory; or
using a copy-on-write functionality of a filesystem to create a clone of the repository into a temporary directory;
performing reads and/or writes of the transaction on the snapshot;
evaluating potential conflicts between the reads and/or the writes of the transaction and those of other concurrently committed transactions;
responsive to no conflict being found, committing the transaction to a write-ahead log; and
updating the repository based on the changes recorded in the write-ahead log.
19. The computer program product of claim 18 , wherein:
committing the transaction to the write-ahead log comprises:
creating a log entry containing the changes to be committed with the transaction; and
persisting the log entry into the write-ahead log; and
updating the repository comprises:
reading a transaction's changes from the write-ahead log; and
based on the changes of the transaction, applying the transaction to the repository by performing the recorded changes.
20. The computer program product of claim 18 , wherein if a crash occurs while one or more transactions have been committed to the write-ahead log but not yet fully applied to the repository, the operations further include recovering committed transactions from the write-ahead log and update the repository accordingly.