Transformer-based AI-module with an activation function

1 . A method for calculating a fully homomorphically encrypted output value of a function of a transformer-based AI-module, dependent on a fully homomorphically encrypted input value of the function, the function representing a non-linear function to be approximated by the function, the method comprising:

providing fully homomorphically encrypted precomputed terms which are independent from the input value; and

calculating the output value of the function, using the transformer-based AI-Module, dependent on the precomputed terms and on polynomials, wherein a respective polynomial depends on a difference between the input value and a center value associated to the respective polynomial, wherein the associated center value indicates a center point at which the respective polynomial is centered and the associated center value differs from polynomial to polynomial.

2 . The method of claim 1 , wherein the respective polynomial depends on a set of powers for the respective polynomial, the powers of the respective set depending each on the difference between the input value and the center value associated to the respective polynomial as a basis of the respective power and on a natural number as an exponent of the respective power, the natural numbers increasing from power to power of the respective polynomial.

3 . The method of claim 2 , wherein the respective polynomial depends on a set of products for the respective polynomial, wherein the respective product of the respective set comprises one of the precomputed terms and one of the powers of the respective set of powers as factors.

4 . The method of claim 2 , wherein the calculating of the output value of the function dependent on the polynomials and the precomputed terms further comprises:

providing a combined polynomial resulting from combining terms having a same natural number as an exponent, wherein the terms result from expanding the respective set of powers of the respective polynomial and arranging the expanded sets of powers together in a form of a single mathematical expression, wherein the combined polynomial comprises the precomputed terms and further terms which depend on the input value and comprise the natural numbers as an exponent;

evaluating the further terms dependent on the input value; and

calculating the output value as a result of evaluating the combined polynomial on the basis of the precomputed terms and the evaluated further terms.

5 . The method of claim 4 , wherein the providing of the combined polynomial further comprises providing fully homomorphically encrypted coefficients of the combined polynomial, wherein the encrypted coefficients of the combined polynomial represent the combined polynomial in a ciphertext domain.

6 . The method of claim 5 , wherein the encrypted coefficients of the combined polynomial are part of the precomputed terms.

7 . The method of claim 1 , wherein the precomputed terms depend on derivatives of the non-linear function evaluated at the center points.

8 . The method of claim 1 , wherein the respective polynomial represents a respective Taylor polynomial for the non-linear function, wherein the respective Taylor polynomial is centered at the respective center value which is associated to the respective polynomial.

9 . The method of claim 1 , wherein the calculating of the output value of the function comprises evaluating a linear combination of the polynomials.

10 . The method of claim 1 , wherein the non-linear function is a Gaussian Error Linear Unit function.

11 . The method of claim 1 , wherein the non-linear function is an exponential function of a softmax function.

12 . The method of claim 1 , wherein the transformer-based AI-Module is a transformer-based Large Language Model.

13 . The method of claim 12 , wherein the function is a function of an attention module of the transformer-based Large Language Model.

14 . The method of claim 1 , the method further comprising:

receiving a first encrypted input;

determining, using the transformer-based AI-Module, fully homomorphically encrypted entries of first key-vectors, first value-vectors, and first query-vectors of an attention module of the transformer-based AI-Module, in a form of fully homomorphically encrypted coefficients of first further polynomials dependent on the first input;

storing a part of the first key-vectors and the first value-vectors, in the form of first tensors, wherein entries of the first tensors comprise the coefficients of a part of the first further polynomials which is associated to the part of the first key-vectors, first value-vectors, and first query-vectors;

determining the input value of the function dependent on the entries of the first vectors;

determining a first output of the transformer-based AI-Module dependent on the output value of the function;

receiving a second encrypted input;

determining, using the transformer-based AI-Module, second key-vectors, second value-vectors, and second query-vectors of the attention module, dependent on the second input;

determining a second input value of the function dependent on the stored first tensors and the second key-vectors, second value-vectors, and second query-vectors;

determining a second output value of the function dependent on the second input value; and

determining a second output of the transformer-based AI-Module dependent on the second output value of the function.

15 . The method of claim 14 , the method further comprising

providing a caching device for storing the first tensors;

dividing the first tensors into chunks of the first tensors; and

storing the chunks in memory blocks of the caching device, wherein the blocks are accessible independently from each other for reading out the chunks of the first tensors independently from each other.

16 . The method of claim 15 , the method further comprising:

allocating the memory blocks non-contiguously in the caching device.

17 . A method for enhancing a transformer-based AI-module, the transformer-based AI-module comprising a non-linear function for calculating an output value of the non-linear function dependent on an input value, the method comprising:

substituting the non-linear function with a function, the function depending on precomputed terms that are independent from the input value and on polynomials, wherein a respective polynomial depends on a difference between the input value and a center value associated to the respective polynomial, wherein the associated center value indicates a center point at which the respective polynomial is centered and the associated center value differs from polynomial to polynomial,

wherein the function is designed to prescribe an approximated output value of the non-linear function in response to presenting the input value as an argument of the non-linear function.

18 . A computer system for calculating a fully homomorphically encrypted output value of a function of an artificial intelligence module, AI-module, in particular of a transformer-based AI-module, dependent on a fully homomorphically encrypted input value of the function, the function representing a non-linear function to be approximated by the function, wherein the computer system is configured to:

provide fully homomorphically encrypted precomputed terms which are independent from the input value,

calculate the output value of the function, using the AI-Module, dependent on the precomputed terms and on polynomials, the respective polynomial depending on a difference between the input value and a center value associated to the respective polynomial, wherein the associated center value indicates a center point at which the respective polynomial is centered and the associated center value differs from polynomial to polynomial.

19 . The computer system of claim 18 , wherein the respective polynomial depending on a set of powers for the respective polynomial, the powers of the respective set depending each on the difference between the input value and the center value associated to the respective polynomial as a basis of the respective power and on a natural number as an exponent of the respective power, the natural numbers increasing from power to power of the respective polynomial.

20 . The computer system of claim 19 , wherein the respective polynomial depends on a set of products for the respective polynomial, wherein the respective product of the respective set comprises one of the precomputed terms and one of the powers of the respective set of powers as factors, wherein the precomputed terms depend on derivatives of the non-linear function evaluated at the center points;

wherein the respective polynomial represents a respective Taylor polynomial for the non-linear function, wherein the respective Taylor polynomial is centered at the respective center value which is associated to the respective polynomial; and

wherein the computer system is configured to calculate the output value of the function through evaluating a linear combination of the polynomials.