Limited deduplication scope for distributed file systems
A method, article of manufacture, and apparatus for limited deduplication scope on a distributed file system is discussed. A write request is received from a client at the metadata server (“MDS”), where the write request comprises a data object identifier and a preferred object store identifier. The MDS determines whether a preferred object store associated with the preferred object store identifier contains a copy of a data object associated with the data object identifier. A write URL comprising the data object identifier and a object store location associated with the preferred object store is transmitted to the client when the preferred object store does not contain the copy of the data object.
1. A method for limited deduplication scope on a distributed file system (“DFS”) in a system that includes a plurality of object stores, the method comprising:
receiving a write request from a client at the metadata server (“MDS”), wherein the write request comprises a data object identifier and a preferred object store identifier;
determining at the MDS whether a preferred object store associated with the preferred object store identifier contains a copy of a data object associated with the data object identifier, wherein the MDS is separate from the preferred object store; and
transmitting a write URL comprising the data object identifier and an object store location associated with the preferred object store to the client when the preferred object store does not contain the copy of the data object, wherein the client writes the data object to the preferred object store using the write URL, wherein communications between the client and the MDS are independent and separate from communications between the client and the preferred object store.
2. The method of claim 1 , further comprising receiving a data object commit from the client.
3. The method of claim 2 , further comprising updating an object store mask on the MDS in response to the data object commit.
4. The method of claim 1 , wherein the data object is stored on a second object store within the DFS, wherein the second object store is not the preferred object store.
5. A computer program product for limited deduplication scope on a distributed file system (“DFS”) in a system that includes a plurality of object stores, the computer program product comprising a non-transitory computer readable medium encoded with computer executable program, the code enabling:
receiving a write request from a client at the metadata server (“MDS”), wherein the write request comprises a data object identifier and a preferred object store identifier;
determining at the MDS whether a preferred object store associated with the preferred object store identifier contains a copy of a data object associated with the data object identifier, wherein the MDS is separate from the preferred object store; and
transmitting a write URL comprising the data object identifier and an object store location associated with the preferred object store to the client when the preferred object store does not contain the copy of the data object, wherein the client writes the data object to the preferred object store using the write URL, wherein communications between the client and the MDS are independent and separate from communications between the client and the preferred object store.
6. The computer program product of claim 5 , further comprising receiving a data object commit from the client.
7. The computer program product of claim 6 , further comprising updating an object store mask on the MDS in response to the data object commit.
8. The computer program product of claim 5 , wherein the data object is stored on a second object store within the DFS, wherein the second object store is not the preferred object store.
9. A system for limited deduplication scope on a distributed file system (“DFS”) in a system that includes a plurality of object stores, the system comprising a computer processor configured to execute instructions comprising:
receiving a write request from a client at the metadata server (“MDS”), wherein the write request comprises a data object identifier and a preferred object store identifier;
determining at the MDS whether a preferred object store associated with the preferred object store identifier contains a copy of a data object associated with the data object identifier, wherein the MDS is separate from the preferred object store; and
transmitting a write URL comprising the data object identifier and an object store location associated with the preferred object store to the client when the preferred object store does not contain the copy of the data object, wherein the client writes the data object to the preferred object store using the write URL, wherein communications between the client and the MDS are independent and separate from communications between the client and the preferred object store.
10. The computer program product of claim 9 , further comprising receiving a data object commit from the client.
11. The computer program product of claim 10 , further comprising updating an object store mask on the MDS in response to the data object commit.
12. The computer program product of claim 9 , wherein the data object is stored on a second object store within the DFS, wherein the second object store is not the preferred object store.