Efficient DVCS storage system
Searching for objects in a distributed version control system. A method includes accessing a single index. The single index indexes objects in a plurality of pack files. Each pack file is a single object container aggregating the content of what would otherwise be loose objects of a version control system within the single object container such that the single object container includes the aggregated content. The method further includes from the single index, determining a location of a version control system object by determining from the index which pack file from among the plurality of pack files the object is stored in, and a location in the pack file where the object is stored. The method also includes retrieving the version control system object from the pack file.
1. A method implemented by a computing system that includes and utilizes one or more hardware processor for searching for objects in a distributed version control system, the method comprising:
the computing system accessing a single index corresponding to a plurality of pack files, the single index indexing objects in a plurality of pack files and including a listing of pack files corresponding to the single index, wherein each pack file is a single object container that includes an aggregated portion of content that would otherwise be loose objects of a version control system, and wherein a soft-cap size corresponding to each pack file in the plurality of pack files is initially set but is later adjustable during run-time, the single index further including metadata for each of the pack files and objects within each of the pack files;
the computing system, from the single index, determining a location of a version control system object by using the single index to identify which pack file from among the plurality of pack files the object is stored in, and a location in the identified pack file where the object is stored; and
the computing system retrieving the version control system object from the identified pack file.
2. The method of claim 1 , further comprising retrieving from the single index the metadata about the identified pack file.
3. The method of claim 2 , wherein the metadata comprises data indicating that objects are sorted in the identified pack file by a time based ordering.
4. The method of claim 2 , wherein the metadata comprises data indicating that objects are sorted in the identified pack file by a name based sorting.
5. The method of claim 2 , wherein the metadata comprises data about transformation or data manipulations applied to the identified pack file.
6. The method of claim 2 , wherein the metadata comprises data about the expiration of the identified pack file so that the identified pack file can be cleaned up.
7. The method of claim 2 , wherein the metadata comprises data indicating that the identified pack file has been repacked.
8. A computer readable hardware storage device having stored thereon computer executable instructions that are executable by one or more processors of a computing system to cause the computing system to:
access a single index for a distributed version control system corresponding to a plurality of pack files, the single index indexing objects that are located among a plurality of pack files and including a listing of pack files corresponding to the single index, wherein each pack file is a single object container that includes an aggregated portion of content that would otherwise be loose objects of a version control system, and wherein a soft-cap size corresponding to each pack file in the plurality of pack files is initially set but is later adjustable during run-time, the single index further including metadata for each of the pack files and objects within each of the pack files;
from the single index, determine a location of a version control system object by using the single index to identify which pack file from among the plurality of pack files the object is stored in, and a location in the identified pack file where the object is stored; and
retrieve the version control system object from the identified pack file.
9. The computer readable hardware storage device of claim 8 , further comprising computer executed instructions that are executable by the one or more processors to cause the computing system to retrieve from the single index the metadata about the identified pack file to be performed.
10. The computer readable hardware storage device of claim 9 , wherein the metadata comprises data indicating that objects are sorted in the identified pack file by a time based ordering.
11. The computer readable hardware storage device of claim 9 , wherein the metadata comprises data indicating that objects are sorted in the identified pack file by a name based sorting.
12. The computer readable hardware storage device of claim 9 , wherein the metadata comprises data about transformation or data manipulations applied to the identified pack file.
13. The computer readable hardware storage device of claim 9 , wherein the metadata comprises data about the expiration of the identified pack file so that the identified pack file can be cleaned up.
14. The computer readable hardware storage device of claim 9 , wherein the metadata comprises data indicating that the identified pack file has been repacked.
15. A version control system, the version control system comprising:
one or more processors;
a data store; and
one or more computer-readable hardware storage devices having stored thereon computer-executable instructions that are executable by the one or more processors and that cause the version control system to instantiate the following:
a plurality of pack files that are stored in the data store, wherein each pack file is a single object container that includes an aggregated portion of content that would otherwise be loose objects of a version control system, and wherein a soft-cap size corresponding to each pack file in the plurality of pack files is initially set but is later adjustable during run-time; and
a single index, wherein the single index indexes objects in the plurality of pack files, the single index including a listing of pack files corresponding to the single index, the single index further including metadata for objects within each of the pack files.
16. The system of claim 15 , wherein the index comprises metadata about each of the pack files in the plurality of pack files.
17. The system of claim 16 , wherein the metadata comprises data indicating that objects are sorted in each of the pack files by a time based ordering.
18. The system of claim 16 , wherein the metadata comprises data indicating that objects are sorted in each of the pack files by a name based sorting.
19. The system of claim 16 , wherein the metadata comprises data about transformations or data manipulations applied to each of the pack files.
20. The system of claim 16 , wherein the metadata comprises data about an expiration that corresponds to each of the pack files so that each of the pack files can be cleaned up according to the corresponding expiration.