Efficient method for data invulnerability architecture (DIA) and data corruption detection for deduped cloud objects
One example method includes receiving, at a cloud storage site, a write request that identifies an object to be written to the cloud storage site, storing the object at the cloud storage site, receiving, at the cloud storage site, a read request that identifies the object, performing, at the cloud storage site, an evaluation of the object to determine if the object is corrupted, and sending a message identifying an outcome of the evaluation. One or more of the operations at the cloud storage site may be performed by an Object Lambda function that has been configured by a dedupe application.
1. A method, comprising:
transmitting, by a dedupe application, a write request to a cloud storage site, and the write request identifies an object to be stored at the cloud storage site;
configuring, by the dedupe application, an Object Lambda function at the cloud storage site to perform an evaluation of the object, and the Object Lambda function, as configured by the dedupe application, includes logic to iterate, parse, and validate, data in the object, and a configuration of the Object Lambda function is determined based on a requirement of the dedupe application;
issuing, by the dedupe application to the cloud storage site, a read request directed to the object;
receiving, by the dedupe application from the cloud storage site, a message, and the message indicates that the evaluation of the object has been performed by the cloud storage site, and the message also indicates an outcome of the evaluation;
when the message indicates a problem with the object, one or more additional write requests are issued by the dedupe application to the cloud storage site until either ‘X’ additional write requests have been made, where ‘X’ is a specified integer that is equal to, or greater than, one, or the problem has been resolved, whichever occurs first; and
when the message does not indicate a problem with the object, no further write requests are issued by the dedupe application to the cloud storage site.
2. The method as recited in claim 1 , wherein the evaluation comprises a comparison of a checksum calculated for the object as received from the dedupe application and a checksum provided by the dedupe application in the read request.
3. The method as recited in claim 1 , wherein the evaluation indicates whether or not the object has been corrupted.
4. The method as recited in claim 1 , wherein the write request identifies object metadata to be stored at the cloud storage site.
5. The method as recited in claim 1 , wherein the message indicates that an evaluation of object metadata has been performed by the cloud storage site.
6. The method as recited in claim 1 , wherein the evaluation was carried out by an Object Lambda function hosted at the cloud storage site.
7. The method as recited in claim 1 , wherein performance of the evaluation is automatically triggered by receipt, at the cloud storage site, of the read request.
8. The method as recited in claim 1 , wherein no data is downloaded from the cloud storage site by the dedupe application.
9. A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
transmitting, by a dedupe application, a write request to a cloud storage site, and the write request identifies an object to be stored at the cloud storage site;
configuring, by the dedupe application, an Object Lambda function at the cloud storage site to perform an evaluation of the object, and the Object Lambda function, as configured by the dedupe application, includes logic to iterate, parse, and validate, data in the object, and a configuration of the Object Lambda function is determined based on a requirement of the dedupe application;
issuing, by the dedupe application to the cloud storage site, a read request directed to the object;
receiving, by the dedupe application from the cloud storage site, a message, and the message indicates that the evaluation of the object has been performed by the cloud storage site, and the message also indicates an outcome of the evaluation;
when the message indicates a problem with the object, one or more additional write requests are issued by the dedupe application to the cloud storage site until either ‘X’ additional write requests have been made, where ‘X’ is a specified integer that is equal to, or greater than, one, or the problem has been resolved, whichever occurs first; and
when the message does not indicate a problem with the object, no further write requests are issued by the dedupe application to the cloud storage site.
10. The non-transitory storage medium as recited in claim 9 , wherein the evaluation comprises a comparison of a checksum calculated for the object as received from the dedupe application and a checksum provided by the dedupe application in the read request.
11. The non-transitory storage medium as recited in claim 9 , wherein the evaluation indicates whether or not the object has been corrupted.
12. The non-transitory storage medium as recited in claim 9 , wherein the write request identifies object metadata to be stored at the cloud storage site.
13. The non-transitory storage medium as recited in claim 9 , wherein the message indicates that an evaluation of object metadata has been performed by the cloud storage site.
14. The non-transitory storage medium as recited in claim 9 , wherein the evaluation was carried out by an Object Lambda function hosted at the cloud storage site.
15. The non-transitory storage medium as recited in claim 9 , wherein performance of the evaluation is automatically triggered by receipt, at the cloud storage site, of the read request.
16. The non-transitory storage medium as recited in claim 9 , wherein no data is downloaded from the cloud storage site by the dedupe application.