Out-of-path, content-addressed writes with untrusted clients
View Patent ↗A method in data storage involves receiving at a metadata server from a client device a BEGIN_WRITE request identifying data to be written by hashes, consulting, by the metadata server a deduplication table to determine historical nature of the data determined to be written, determining that the data was never previously written, was previously written by another client, or was previously written but failed a verification, sending to the client device write locations with tokens for the data, starting an authentication timer, and authorizing the client device to write directly to the data server using the authentication tokens and the authenticated write locations, determining that the data was previously written by the same client, or that the data was previously written and passed the verification test, notifying the client of this circumstance, and receiving from the client device an END_WRITE request.
1. A method for writing client data to a data server using a metadata server, the method comprising:
receiving at the metadata server a BEGIN_WRITE request from a client, wherein the BEGIN_WRITE request includes a plurality of hashes of data to be written to the data server;
consulting a deduplication table to determine the data has not previously been written to the data server;
transmitting both a list of data locations on the data server and a set of authentication tokens for the data locations to the client, wherein the authentication tokens authorize the client to write the data identified by the hashes to the data locations, wherein the authorization tokens are associated with an authorization timeout;
receiving at the metadata server an END_WRITE request, wherein the END_WRITE request indicates the client has finished writing the data to the data server;
subjecting the data written to the data server to a quarantine, wherein additional clients can not write to the data locations during the quarantine;
verifying from the metadata server that the data is present on the data server when the authorization timeout has expired; and
removing the quarantine in response to a passing verification.
2. A system for writing client data to a data server using a metadata server comprising a non-transitory computer readable storage medium and a processor executing instructions for:
receiving at the metadata server a BEGIN_WRITE request from a client, wherein the BEGIN_WRITE request includes a plurality of hashes of data to be written to the data server;
consulting a deduplication table to determine the data has not previously been written to the data server;
transmitting both a list of data locations on the data server and a set of authentication tokens for the data locations to the client, wherein the authentication tokens authorize the client to write the data identified by the hashes to the data locations, wherein the authorization tokens are associated with an authorization timeout;
receiving at the metadata server an END_WRITE request, wherein the END_WRITE request indicates the client has finished writing the data to the data server;
subjecting the data written to the data server to a quarantine, wherein additional clients can not write to the data locations during the quarantine;
verifying from the metadata server that the data is present on the data server when the authorization timeout has expired; and
removing the quarantine in response to a passing verification.
3. A computer program product for writing client data to a data server using a metadata server, the computer program product comprising a non-transitory computer readable medium encoded with computer executable program, the code enabling:
receiving at the metadata server a BEGIN_WRITE request from a client, wherein the BEGIN_WRITE request includes a plurality of hashes of data to be written to the data server;
consulting a deduplication table to determine the data has not previously been written to the data server;
transmitting both a list of data locations on the data server and a set of authentication tokens for the data locations to the client, wherein the authentication tokens authorize the client to write the data identified by the hashes to the data locations, wherein the authorization tokens are associated with an authorization timeout;
receiving at the metadata server an END_WRITE request, wherein the END_WRITE request indicates the client has finished writing the data to the data server;
subjecting the data written to the data server to a quarantine, wherein additional clients can not write to the data locations during the quarantine;
verifying from the metadata server that the data is present on the data server when the authorization timeout has expired; and
removing the quarantine in response to a passing verification.