Data protection for a document database system

1. A method of data protection in a document database system, comprising:

placing documents input to the document database system into same sized chunks of super documents such that each of the chunks of the super documents includes one of the documents and a header that includes document metadata of the one of the documents, wherein the documents are NoSQL database documents, wherein the documents have different sizes and at least some of the plurality of chunks are padded with padding in addition to a unique document so that each of the chunks has the same size;

generating and storing metadata for the super documents in a super document index, wherein the metadata for each of the super documents includes identifiers for the documents included in that super document;

sending each of the super documents to an object storage system for parity processing according to a storage policy, wherein the parity processing includes computing a plurality of parity chunks for each of the super documents and storing the parity chunks in an object store;

querying the super document index to obtain particular metadata for associated super documents; and

processing the associated super documents to update or restore at least some of the documents in the associated super documents using the parity chunks of the associated super documents stored in the object store.

2. The method of claim 1 , wherein the documents are stored in a document store managed by the document database system.

3. The method of claim 1 , wherein the document database system includes a plurality of shards, and wherein the plurality of chunks for each of the super documents includes documents from at least one of the plurality of shards, wherein each of the shards maintains a unique set of documents.

4. The method of claim 1 , further comprising:

providing the metadata to the object storage system for association with the parity chunks.

5. The method of claim 1 , wherein the document database system includes a plurality of shards, wherein each of the shards maintains a unique set of documents, the method further comprising:

querying the super document index to obtain selected metadata based on a shard identifier for a failed shard of the plurality of shards;

retrieving selected documents from operating shards of the plurality of shards based on the selected metadata;

retrieving selected parity chunks from the object storage system based on the selected metadata; and

restoring lost documents managed by the failed shard from the selected documents and the selected parity chunks.

6. The method of claim 1 , further comprising:

querying the super document index to obtain selected metadata of a selected super document based on a document identifier;

modifying a selected super document based on the selected metadata to delete a document from the selected super document or to update a document in the selected super document; and

sending the selected super document to the object storage system to update the plurality of parity chunks associated with the selected super document based on the modified super document.

7. A computer system, comprising:

a memory configured to store software; and

a processor configured to execute the software to implement a listener in communication with a document database system and an object storage system, the listener configured to:

place documents input to the document database system into same sized chunks of super documents such that each of the chunks of the super documents includes one of the documents and a header that includes document metadata of the one of the documents, wherein the documents are NoSQL database documents, wherein the documents have different sizes and at least some of the plurality of chunks are padded with padding in addition to a unique document so that each of the chunks has the same size;

generate and store metadata for the super documents in a super document index, wherein the metadata for each of the super documents includes identifiers for the documents included in that super document;

send each of the super documents to the object storage system for parity processing according to a storage policy, where the parity processing includes computing a plurality of parity chunks for each of the super documents and storing the parity chunks in an object store;

query the super document index to obtain particular metadata for associated super documents; and

process the associated super documents to update or restore at least some of the documents in the associated super documents using the parity chunks of the associated super documents stored in the object store.

8. The computer system of claim 7 , wherein the documents are stored in a document store managed by the document database system.

9. The computer system of claim 7 , wherein the document database system includes a plurality of shards, and wherein the plurality of chunks for each of the super documents includes documents from at least one of the plurality of shards, wherein each of the shards maintains a unique set of documents.

10. The computer system of claim 7 , wherein the listener is further configured to:

provide the metadata to the object storage system for association with the parity chunks.

11. The computer system of claim 8 , wherein the document database system includes a plurality of shards, wherein each of the shards maintains a unique set of documents, the listener is further configured to:

query the super document index to obtain selected metadata based on a shard identifier for a failed shard of the plurality of shards;

retrieve selected documents from operating shards of the plurality of shards based on the selected metadata;

retrieve selected parity chunks from the object storage system based on the selected metadata; and

restore lost documents managed by the failed shard from the selected documents and the selected parity chunks.

12. The computer system of claim 8 , wherein the listener is further configured to:

query the super document index to obtain selected metadata of a selected super document based on a document identifier;

modify a selected super document based on the selected metadata to delete a document from the selected super document or to update a document in the selected super document; and

send the selected super document to the object storage system to update the plurality of parity chunks associated with the selected super document based on the modified super document.

13. A non-transitory computer readable medium comprising instructions, which when executed in a computer system, causes the computer system to carry out a method of data protection in a document database system, comprising:

placing documents input to the document database system into same sized chunks of super documents such that each of the chunks of the super documents includes one of the documents and a header that includes document metadata of the one of the documents, wherein the documents are NoSQL database documents, wherein the documents have different sizes and at least some of the plurality of chunks are padded with padding in addition to a unique document so that each of the chunks has the same size;

generating and storing metadata for the super documents in a super document index, wherein the metadata for each of the super documents includes identifiers for the documents included in that super document;

sending each of the super documents to an object storage system for parity processing according to a storage policy, where the parity processing includes computing a plurality of parity chunks for each of the super documents and storing the parity chunks in an object store;

querying the super document index to obtain particular metadata for associated super documents; and

processing the associated super documents to update or restore at least some of the documents in the associated super documents using the parity chunks of the associated super documents stored in the object store.

14. The non-transitory computer readable medium of claim 13 , wherein the documents are stored in a document store managed by the document database system.

15. The non-transitory computer readable medium of claim 13 , wherein the document database system includes a plurality of shards, and wherein the plurality of chunks for each of the super documents includes documents from at least one of the plurality of shards, wherein each of the shards maintains a unique set of documents.

16. The non-transitory computer readable medium of claim 13 , wherein the document database system includes a plurality of shards, wherein each of the shards maintains a unique set of documents, the method further comprising:

querying the super document index to obtain selected metadata based on a shard identifier for a failed shard of the plurality of shards;

retrieving selected documents from operating shards of the plurality of shards based on the selected metadata;

retrieving selected parity chunks from the object storage system based on the selected metadata; and

restoring lost documents managed by the failed shard from the selected documents and the selected parity chunks.

17. The non-transitory computer readable medium of claim 13 , further comprising:

querying the super document index to obtain selected metadata of a selected super document based on a document identifier;

modifying a selected super document based on the selected metadata to delete a document from the selected super document or to update a document in the selected super document; and

sending the selected super document to the object storage system to update the plurality of parity chunks associated with the selected super document based on the modified super document.