Method and apparatus of securely processing data for file backup, de-duplication, and restoration
View Patent ↗Disclosed are an apparatus and method of de-duplicating at least one data file. One example method may include requesting a list of data rows stored in a database to be de-duplicated, receiving the list of the data rows based on a single filekey associated with the at least one data file, copying the at least one data file to a data storage memory, and deleting the data rows entries from a file registry of the database.
1. A method of de-duplicating at least one data file, the method comprising:
requesting a list of data rows stored in a database to be de-duplicated;
receiving the list of the data rows based on a single filekey associated with the at least one data file;
copying the at least one data file to a data storage memory;
deleting the data rows entries from a file registry of the database;
determining if the filekey has already been de-duplicated;
decrypting an advanced encryption standard (AES) key associated with each of the rows in the list of data rows; and
re-encrypting the AES key to transfer the AES key to an agent application.
2. The method of claim 1 , wherein the list of the data rows comprises a maximum number of data rows.
3. The method of claim 1 , wherein the requesting, receiving, copying and deleting operations are performed by a crawler application.
4. The method of claim 1 , further comprising:
if the filekey has not already been de-duplicated, copying the at least one data file and metadata to a data repository bucket.
5. The method of claim 1 , further comprising:
creating a link file in place of the at least one data file including a uniform resource identifier (URI) and the re-encrypted AES key; and
storing the link file in an online memory storage.
6. The method of claim 5 , wherein the URI of the link file and the re-encrypted AES key are stored in metadata of the link file.
7. An apparatus configured to de-duplicate at least one data file, the apparatus comprising:
a transmitter configured to request a list of data rows stored in a database to be de-duplicated;
a receiver configured to receive the list of the data rows based on a single filekey associated with the at least one data file; and
a processor configured to
copy the at least one data file to a data storage memory,
delete the data rows entries from a file registry of the database,
determine if the filekey has already been de-duplicated,
decrypt an advanced encryption standard (AES) key associated with each of the rows in the list of data rows, and
re-encrypt the AES key to transfer the AES key to an agent application.
8. The apparatus of claim 7 , wherein the list of the data rows comprises a maximum number of data rows.
9. The apparatus of claim 7 , wherein the request, receive, copy and delete operations are performed by a crawler application installed in a memory of the apparatus.
10. The apparatus of claim 7 , wherein the processor is further configured to if the filekey has not already been de-duplicated, copy the at least one data file and metadata to a data repository bucket.
11. The apparatus of claim 7 , wherein the processor is further configured to create a link file in place of the at least one data file including a uniform resource identifier (URI) and the re-encrypted AES key, and store the link file in an online memory storage.
12. The apparatus of claim 11 , wherein the URI of the link file and the re-encrypted AES key are stored in metadata of the link file.
13. A non-transitory computer readable storage medium configured to store instructions that when executed causes a processor to perform de-duplicating at least one data file, the processor being configured to perform:
requesting a list of data rows stored in a database to be de-duplicated;
receiving the list of the data rows based on a single filekey associated with the at least one data file;
copying the at least one data file to a data storage memory;
deleting the data rows entries from a file registry of the database:,
determining if the filekey has already been de-duplicated;
decrypting an advanced encryption standard (AES) key associated with each of the rows in the list of data rows; and
re-encrypting the AES key to transfer the AES key to an agent application.
14. The non-transitory computer readable storage medium of claim 13 , wherein the list of the data rows comprises a maximum number of data rows.
15. The non-transitory computer readable storage medium of claim 13 , wherein the requesting, receiving, copying and deleting operations are performed by a crawler application.
16. The non-transitory computer readable storage medium of claim 13 , wherein the processor is further configured to perform:
if the filekey has already been de-duplicated, copying the at least one data file and metadata to a data repository bucket.
17. The non-transitory computer readable storage medium of claim 13 , wherein the processor is further configured to perform:
creating a link file in place of the at least one data file including a uniform resource identifier (URI) and the re-encrypted AES key; and
storing the link file in an online memory storage.