Fully homomorphic encryption (FHE) and partially homomorphic encryption (PHE) in distributed 1-bit large language model (LLM) architecture

1 . A method for securely executing a machine learning model (MLM), the method comprising:

determining whether a first operation performed by an MLM is compatible with one of a first encryption scheme and a second encryption scheme, wherein the MLM is distributed over at least one client device and at least one server, and wherein determining whether the first operation is compatible with the first encryption scheme comprises determining whether the first operation can be reduced to one or more addition operation;

in response to determining that the first operation can be reduced to one or more addition operations:

encrypting data associated with the first operation using the first encryption scheme; and

transmitting the data encrypted by the first encryption scheme to the at least one server configured to apply the first operation; and

in response to determining that the first operation is incompatible with the first encryption scheme:

encrypting the data associated with the first operation using the second encryption scheme; and

transmitting the data encrypted by the second encryption scheme to the at least one server configured to apply the first operation.

2 . The method of claim 1 , wherein the first encryption scheme is partially homomorphic encryption (PHE) and the second encryption scheme is fully homomorphic encryption (FHE).

3 . The method of claim 1 , wherein the MLM is a 1-bit large language model (LLM).

4 . The method of claim 1 , wherein the data encrypted by the first encryption scheme is transmitted to a first server of the at least one server and the data encrypted by the second encryption scheme is transmitted to a second server of the at least one server.

5 . The method of claim 4 , wherein the first encryption scheme utilizes fewer computational resources than the second encryption scheme, and wherein the data encrypted by the first encryption scheme and the data encrypted by the second encryption scheme are respectively transmitted to the first server and the second server in response to determining that the second server comprises more computational resources than the first server.

6 . The method of claim 1 , wherein the data is input data provided by a user, further comprising:

receiving, by the at least one client device, a result of the first operation from the at least one server; and

determining a decrypted value from the result using a decryption key associated with one of the first encryption scheme and the second encryption scheme.

7 . The method of claim 6 , further comprising:

outputting the decrypted value on the at least one client device.

8 . The method of claim 6 , wherein the result is encrypted using the first encryption scheme, further comprising:

determining whether a second operation performed by the MLM is compatible with the first encryption scheme; and

in response to determining that the second operation is incompatible with the first encryption scheme, encrypting the decrypted value using the second encryption scheme for application of the second operation using the at least one server.

9 . The method of claim 1 , wherein the first operation comprises matrix multiplication, wherein the first encryption scheme is applied on the data.

10 . A system for securely executing a machine learning model (MLM), comprising:

at least one memory; and

at least one hardware processor coupled with the at least one memory and configured, individually or in combination, to:

determine whether a first operation performed by an MLM is compatible with one of a first encryption scheme and a second encryption scheme, wherein the MLM is distributed over at least one client device and at least one server, and wherein determining whether the first operation is compatible with the first encryption scheme comprises determining whether the first operation can be reduced to one or more addition operation;

in response to determining that the first operation can be reduced to one or more addition operations:

encrypt data associated with the first operation using the first encryption scheme; and

transmit the data encrypted by the first encryption scheme to the at least one server configured to apply the first operation; and

in response to determining that the first operation is incompatible with the first encryption scheme:

encrypt the data associated with the first operation using the second encryption scheme; and

transmit the data encrypted by the second encryption scheme to the at least one server configured to apply the first operation.

11 . The system of claim 10 , wherein the first encryption scheme is partially homomorphic encryption (PHE) and the second encryption scheme is fully homomorphic encryption (FHE).

12 . The system of claim 10 , wherein the MLM is a 1-bit large language model (LLM).

13 . The system of claim 10 , wherein the data encrypted by the first encryption scheme is transmitted to a first server of the at least one server and the data encrypted by the second encryption scheme is transmitted to a second server of the at least one server.

14 . The system of claim 13 , wherein the first encryption scheme utilizes fewer computational resources than the second encryption scheme, and wherein the data encrypted by the first encryption scheme and the data encrypted by the second encryption scheme are respectively transmitted to the first server and the second server in response to determining that the second server comprises more computational resources than the first server.

15 . The system of claim 10 , wherein the data is input data provided by a user, wherein the at least one hardware processor is configured to:

receive, by the at least one client device, a result of the first operation from the at least one server; and

determine a decrypted value from the result using a decryption key associated with one of the first encryption scheme and the second encryption scheme.

16 . The system of claim 15 , wherein the at least one hardware processor is configured to:

output the decrypted value on the at least one client device.

17 . The system of claim 15 , wherein the result is encrypted using the first encryption scheme, wherein the at least one hardware processor is configured to:

determine whether a second operation performed by the MLM is compatible with the first encryption scheme; and

in response to determining that the second operation is incompatible with the first encryption scheme, encrypt the decrypted value using the second encryption scheme for application of the second operation using the at least one server.

18 . A non-transitory computer readable medium storing thereon computer executable instructions for securely executing a machine learning model (MLM), including instructions for:

determining whether a first operation performed by an MLM is compatible with one of a first encryption scheme and a second encryption scheme, wherein the MLM is distributed over at least one client device and at least one server, and wherein determining whether the first operation is compatible with the first encryption scheme comprises determining whether the first operation can be reduced to one or more addition operation;

in response to determining that the first operation can be reduced to one or more addition operations:

encrypting data associated with the first operation using the first encryption scheme; and

transmitting the data encrypted by the first encryption scheme to the at least one server configured to apply the first operation; and

in response to determining that the first operation is incompatible with the first encryption scheme:

encrypting the data associated with the first operation using the second encryption scheme; and

transmitting the data encrypted by the second encryption scheme to the at least one server configured to apply the first operation.