Data lakehouse encryption
Examples described herein provide a computer-implemented method for data lakehouse encryption that includes receiving a query for data stored in a data lakehouse. The method further includes determining whether the query is a frequent query by comparing a frequency of the query to a threshold. The method further includes, responsive to determining that the query is a frequent query, storing envelope encrypted data and a query index in a frequent data access database, the envelope encrypted data being data that relates to the query. The method further includes, responsive to receiving the query at a later time, returning, to a user, the envelope encrypted data from the frequent data access database without decrypting other data stored in the data lakehouse.
1 . A computer-implemented method for data lakehouse encryption, the method comprising:
receiving a query for data stored in a data lakehouse;
determining whether the query is a frequent query by comparing a frequency of the query to a threshold;
responsive to determining that the query is a frequent query, storing envelope encrypted data and a query index in a frequent data access database, the envelope encrypted data being data that relates to the query;
responsive to receiving the query at a later time, returning, to a user, the envelope encrypted data from the frequent data access database without decrypting other data stored in the data lakehouse;
monitoring a rate at which the envelope encrypted data in the frequent data access database are queried, and, responsive to the rate being below a threshold rate, removing the envelope encrypted data from the frequent data access database; and
responsive to the rate being below the threshold rate, moving the envelope encrypted data from the frequent data access database and the data from the data lakehouse to an archive.
2 . The computer-implemented method of claim 1 , wherein it is determined that the query is a frequent query responsive to the frequency of the query satisfying the threshold.
3 . The computer-implemented method of claim 2 , wherein the threshold is based on a predefined number of repetitions of the query within a time period.
4 . The computer-implemented method of claim 1 , wherein the envelope encrypted data is encrypted by performing envelope encryption, which comprises encrypting the data with a data encryption key and then encrypting the data encryption key with a key encryption key to encrypt the data.
5 . The computer-implemented method of claim 1 , wherein the threshold is dynamically adjusted.
6 . The computer-implemented method of claim 1 , further comprising updating the frequent data access database in response to any updates or edits to the envelope encrypted data.
7 . A computer system comprising:
a processor set;
one or more computer-readable storage media; and
program instructions stored on the one or more computer-readable storage media to cause the processor set to perform operations comprising:
receiving a query for data stored in a data lakehouse;
determining whether the query is a frequent query by comparing a frequency of the query to a threshold;
responsive to determining that the query is a frequent query, storing envelope encrypted data and a query index in a frequent data access database, the envelope encrypted data being data that relates to the query;
responsive to receiving the query at a later time, returning, to a user, the envelope encrypted data from the frequent data access database without decrypting other data stored in the data lakehouse;
monitoring a rate at which the envelope encrypted data in the frequent data access database are queried, and, responsive to the rate being below a threshold rate, removing the envelope encrypted data from the frequent data access database; and
responsive to the rate being below the threshold rate, moving the envelope encrypted data from the frequent data access database and the data from the data lakehouse to an archive.
8 . The computer system of claim 7 , wherein it is determined that the query is a frequent query responsive to the frequency of the query satisfying the threshold.
9 . The computer system of claim 8 , wherein the threshold is based on a predefined number of repetitions of the query within a time period.
10 . The computer system of claim 7 , wherein the envelope encrypted data is encrypted by performing envelope encryption, which comprises encrypting the data with a data encryption key and then encrypting the data encryption key with a key encryption key to encrypt the data.
11 . The computer system of claim 7 , wherein the threshold is dynamically adjusted.
12 . The computer system of claim 7 , wherein the operations further comprise updating the frequent data access database in response to any updates or edits to the envelope encrypted data.
13 . A computer program product comprising:
one or more computer-readable storage media; and
program instructions stored on the one or more computer-readable storage media to perform operations comprising:
receiving a query for data stored in a data lakehouse;
determining whether the query is a frequent query by comparing a frequency of the query to a threshold;
responsive to determining that the query is a frequent query, storing envelope encrypted data and a query index in a frequent data access database, the envelope encrypted data being data that relates to the query;
responsive to receiving the query at a later time, returning, to a user, the envelope encrypted data from the frequent data access database without decrypting other data stored in the data lakehouse;
monitoring a rate at which the envelope encrypted data in the frequent data access database are queried, and, responsive to the rate being below a threshold rate, removing the envelope encrypted data from the frequent data access database; and
responsive to the rate being below the threshold rate, moving the envelope encrypted data from the frequent data access database and the data from the data lakehouse to an archive.
14 . The computer program product of claim 13 , wherein it is determined that the query is a frequent query responsive to the frequency of the query satisfying the threshold.
15 . The computer program product of claim 14 , wherein the threshold is based on a predefined number of repetitions of the query within a time period.
16 . The computer program product of claim 13 , wherein the envelope encrypted data is encrypted by performing envelope encryption, which comprises encrypting the data with a data encryption key and then encrypting the data encryption key with a key encryption key to encrypt the data.