Systems and methods for maintaining encrypted search indexes on third-party storage systems
A computer-implemented method for maintaining encrypted search indexes on third party storage systems may include (1) identifying a plurality of encrypted files, (2) identifying a plurality of keywords contained in the plurality of encrypted files, and (3) generating an encrypted search index for searching the plurality of encrypted files by (i) identifying, for each keyword in the plurality of keywords, a list of encrypted files in the plurality of encrypted files that contain the keyword, (ii) encrypting the list of encrypted files, and (iii) storing the encrypted list of encrypted files such that the encrypted list of encrypted files can be identified using a lookup key generated by applying a pseudo-random function to the keyword. Various other methods, systems, and computer-readable media are also disclosed.
1. A computer-implemented method for maintaining encrypted search indexes on third party storage systems, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:
identifying, as part of a plugin of a search engine, a dynamic search index used by the search engine, wherein:
the dynamic search index comprises a plurality of data chunks;
each of the plurality of data chunks comprises one or more data blocks;
each of the plurality of data chunks has been encrypted using a unique nonce; and
a search-index initialization vector is designated and stored for encrypting the dynamic search index; and
enabling a single data chunk in the plurality of data chunks to be accessed by decrypting, as part of the plugin of the search engine, the single data chunk by:
calculating a chunk initialization vector for decrypting the single data chunk by:
identifying the unique nonce used to encrypt the single data chunk; and
deriving the chunk initialization vector by summing the search-index initialization vector with a product of the unique nonce and a number of the one or more data blocks; and
using the chunk initialization vector to decrypt the single data chunk; and
reencrypting, as part of the plugin of the search engine after the single data chunk has been accessed, the single data chunk by:
calculating a new chunk initialization vector for encrypting the single data chunk such that no two data chunks in the plurality of data chunks have identical initialization vectors by:
calculating a new unique nonce for the single data chunk; and
deriving the new chunk initialization vector by summing the search-index initialization vector with a product of the new unique nonce and the number of the one or more data blocks; and
using the new chunk initialization vector to encrypt the single data chunk.
2. The method of claim 1 , wherein a size of each of the plurality of data chunks is equal to a size of a block of data that the search engine writes to the dynamic search index in a single write operation.
3. The method of claim 1 , wherein a size of each of the one or more data blocks of each of the plurality of data chunks is equal to a size of a block of data that is encrypted by a block cipher in a single operation.
4. The method of claim 1 , wherein the plugin of the search engine comprises at least one of:
an addon; and
an extention.
5. The method of claim 1 , wherein enabling the single data chunk to be accessed comprises:
receiving a request from the search engine for the single data chunk; and
individually decrypting the single data chunk without decrypting any other data chunk in the plurality of data chunks.
6. The method of claim 1 , wherein unique nonces of the plurality of data chunks are calculated using a global counter such that no nonce is used more than once.
7. The method of claim 1 , wherein an additional new unique nonce is generated for the single data chunk each time the single data chunk is reencrypted.
8. The method of claim 1 , further comprising:
encrypting the new unique nonce using format-preserving encryption; and
storing the encrypted new unique nonce.
9. The method of claim 1 , wherein the new unique nonce is smaller than the chunk initialization vector.
10. The method of claim 1 , wherein:
using the chunk initialization vector to encrypt the single data chunk comprises encrypting the single data chunk using a mode of operation with a randomized counter; and
the randomized counter is generated based at least in part on the chunk initialization vector.
11. The method of claim 10 , further comprising ensuring that each value in a counter range used to encrypt the single data chunk is used only once.
12. A system for maintaining encrypted dynamic search indexes on third party storage systems, the system comprising:
a search-index identifying module, stored in memory, that identifies, as part of a plugin of a search engine, a dynamic search index used by the search engine, wherein:
the dynamic search index comprises a plurality of data chunks;
each of the plurality of data chunks comprises one or more data blocks;
each of the plurality of data chunks has been encrypted using a unique nonce; and
a search-index initialization vector is designated and stored for encrypting the dynamic search index;
a decrypting module, stored in memory, that enables a single data chunk in the plurality of data chunks to be accessed by decrypting, as part of the plugin of the search engine, the single data chunk by:
calculating a chunk initialization vector for decrypting the single data chunk by:
identifying the unique nonce used to encrypt the single data chunk; and
deriving the chunk initialization vector by summing the search-index initialization vector with a product of the unique nonce and a number of the one or more data blocks; and
using the chunk initialization vector to decrypt the single data chunk;
a search-index encrypting module, stored in memory, that reencrypts, as part of the plugin of the search engine after the single data chunk has been accessed, the single data chunk by:
calculating a new chunk initialization vector for encrypting the single data chunk such that no two data chunks in the plurality of data chunks have identical initialization vectors by:
calculating a new unique nonce for the single data chunk; and
deriving the new chunk initialization vector by summing the search-index initialization vector with a product of the new unique nonce and the number of the one or more data blocks; and
using the new chunk initialization vector to encrypt the single data chunk; and
at least one processor that executes the search-index identifying module and the search-index encrypting module.
13. The system of claim 12 , wherein a size of each of the plurality of data chunks is equal to a size of a block of data that the search engine writes to the dynamic search index in a single write operation.
14. The system of claim 12 , wherein a size of each of the one or more data blocks of each of the plurality of data chunks is equal to a size of a block of data that is encrypted by a block cipher in a single operation.
15. The system of claim 12 , wherein the plugin of the search engine comprises at least one of:
an addon; and
an extention.
16. The system of claim 12 , wherein the decrypting module enables the single data chunk to be accessed by:
receiving a request from the search engine for the single data chunk; and
individually decrypting the single data chunk without decrypting any other data chunk in the plurality of data chunks.
17. The system of claim 12 , wherein the search-index encrypting module calculates unique nonces of the plurality of data chunks using a global counter such that no nonce is used more than once.
18. The system of claim 12 , wherein the search-index encrypting module generates an additional new unique nonce for the single data chunk each time the single data chunk is reencrypted.
19. The system of claim 12 , wherein the search-index encrypting module further:
encrypts the new unique nonce using format-preserving encryption; and
stores the encrypted new unique nonce.
20. A non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
identify, as part of a plugin of a search engine, a dynamic search index used by the search engine, wherein:
the dynamic search index comprises a plurality of data chunks;
each of the plurality of data chunks comprises one or more data blocks;
each of the plurality of data chunks has been encrypted using a unique nonce; and
a search-index initialization vector is designated and stored for encrypting the dynamic search index; and
enable a single data chunk in the plurality of data chunks to be accessed by decrypting, as part of the plugin of the search engine, the single data chunk by:
calculating a chunk initialization vector for decrypting the single data chunk by:
identifying the unique nonce used to encrypt the single data chunk; and
deriving the chunk initialization vector by summing the search-index initialization vector with a product of the unique nonce and a number of the one or more data blocks; and
using the chunk initialization vector to decrypt the single data chunk; and
reencrypt, as part of the plugin of the search engine after the single data chunk has been accessed, the single data chunk by:
calculating a new chunk initialization vector for encrypting the single data chunk such that no two data chunks in the plurality of data chunks have identical initialization vectors by:
calculating a new unique nonce for the single data chunk; and
deriving the new chunk initialization vector by summing the search-index initialization vector with a product of the new unique nonce and the number of the one or more data blocks; and
using the new chunk initialization vector to encrypt the single data chunk.