Private inference for generative artificial intelligence systems

1 . A computer-implemented method comprising:

receiving, from a client device in communication with a generative artificial intelligence (AI) system, an encrypted chat message marked with a private status;

generating a decrypted chat message by decrypting the encrypted chat message using a shared session key established at the client device based on a hardware-backed attestation indicating server-side processing by a trusted execution environment;

generating, using a secure enclave associated with the trusted execution environment, a re-encrypted chat message by re-encrypting the decrypted chat message with an additional session key established for a private engine instance; and

utilizing the private engine instance operating in a confidential container to decrypt the re-encrypted chat message and initiate a generative model to generate a response corresponding to the encrypted chat message marked with the private status.

2 . The computer-implemented method of claim 1 , wherein utilizing the private engine instance comprises executing a large language model within the trusted execution environment to generate a plaintext response based on plaintext tokens derived from decrypting the re-encrypted chat message, wherein the plaintext tokens and the plaintext response are inaccessible to the generative AI system outside of the trusted execution environment.

3 . The computer-implemented method of claim 1 , further comprising:

encrypting, using a client-controlled storage encryption key, at least a portion of the response for storage as encrypted chat history; and

storing the encrypted chat history on a server without access to the client-controlled storage encryption key.

4 . The computer-implemented method of claim 1 , further comprising providing, by the trusted execution environment, the hardware-backed attestation to the client device as a cryptographic attestation indicating that the trusted execution environment is executing an audited version of server-side code.

5 . The computer-implemented method of claim 1 , further comprising:

receiving verification from the client device that the hardware-backed attestation is satisfactory for establishing a private chat between the client device and the generative AI system; and

in response to receiving the verification, establishing the shared session key with the client device.

6 . The computer-implemented method of claim 1 , further comprising:

executing, within the trusted execution environment, a safety classification operation to determine a safety classification for plaintext of the chat message or the response;

based on the safety classification corresponding to a predefined safety condition, applying one or more restrictions to a user account associated with the client device, the one or more restrictions including at least one of suspending private chat functionality, rate-limiting subsequent requests, or disabling access to the generative AI system; and

providing, for display via a graphical user interface of the client device, a notification indicating that a chat session associated with the chat message has been terminated.

7 . The computer-implemented method of claim 1 , further comprising generating synthetic data using the generative artificial intelligence system by:

aggregating, within the trusted execution environment, plaintext of a plurality of chat messages satisfying a predefined similarity threshold;

generating, using a large language model within the trusted execution environment, a synthetic message resembling the plurality of chat messages, while applying a differential privacy mechanism to limit contribution of individual chat messages from the plurality of chat messages to the generation of the synthetic message; and

exporting the synthetic message to an external portion of the generative AI system outside of the trusted execution environment.

8 . The computer-implemented method of claim 1 , further comprising transitioning a conversation from a private processing mode to a non-private processing mode by:

determining that subsequent processing of the conversation is not eligible for execution within the trusted execution environment;

providing, for display via a graphical user interface of the client device, a notification indicating that the subsequent processing will continue outside of the trusted execution environment; and

based on receiving an input from the client device indicating consent to transition to the non-private processing mode, processing subsequent chat messages or operations of the conversation outside of the trusted execution environment.

9 . The computer-implemented method of claim 1 , wherein:

receiving the encrypted chat message comprises receiving encrypted multimodal input including at least one of image data, audio data, or video data; and

utilizing the private engine instance comprises executing the generative model within the trusted execution environment to process the multimodal input including at least one of the image data, the audio data, or the video data.

10 . A system comprising:

at least one processor; and

a non-transitory computer-readable storage medium comprising instructions that, when executed by the at least one processor, cause the system to perform operations comprising:

receiving, from a client device in communication with a generative artificial intelligence (AI) system, an encrypted chat message marked with a private status;

generating a decrypted chat message by decrypting, within a trusted execution environment, the encrypted chat message using a first shared session key established between the client device and a server based on validation of a hardware-backed attestation indicating execution of trusted code within the trusted execution environment;

generating a re-encrypted chat message by re-encrypting, within the trusted execution environment, the decrypted chat message using a second session key established for secure communication with a private engine instance; and

utilizing the private engine instance operating within a confidential container to decrypt the re-encrypted chat message and initiate a generative model to generate a response corresponding to the encrypted chat message marked with the private status.

11 . The system of claim 10 , wherein the operations further comprise:

encrypting, using a client-controlled storage encryption key, response content for storage as encrypted chat history; and

storing, on a server without access to the client-controlled storage encryption key, the encrypted chat history for subsequent retrieval of the response content.

12 . The system of claim 10 , wherein the operations further comprise:

providing, by the trusted execution environment, the hardware-backed attestation to the client device indicating execution of an audited version of server-side code within the trusted execution environment;

receiving confirmation from the client device that the hardware-backed attestation satisfies a trust criterion for private processing; and

establishing the first shared session key after receiving the confirmation.

13 . The system of claim 10 , wherein the operations further comprise:

executing, within the trusted execution environment, a safety classification operation on plaintext associated with the encrypted chat message or the response; and

based on the safety classification corresponding to a predefined safety condition, applying one or more restrictions to a user account associated with the client device, the one or more restrictions including at least one of suspending private chat functionality, rate-limiting subsequent requests, or disabling access to the generative AI system.

14 . The system of claim 10 , wherein the operations further comprise generating synthetic data by:

aggregating, within the trusted execution environment, plaintext from a plurality of chat messages satisfying a similarity criterion;

executing a large language model within the trusted execution environment to produce a synthetic message resembling the plurality of chat messages while applying a differential privacy mechanism to limit contribution of individual chat messages; and

exporting the synthetic message for use outside the trusted execution environment.

15 . A non-transitory computer-readable storage medium comprising instructions that, when executed by at least one processor, cause a computing device to perform operations comprising:

receiving, from a client device in communication with a generative artificial intelligence (AI) system, an encrypted chat message marked with a private status;

decrypting, within a trusted execution environment, the encrypted chat message using a first session key shared between the client device and a server;

re-encrypting, within a secure enclave implemented by the trusted execution environment, the decrypted chat message using a second session key established for communication with a private engine instance; and

utilizing the private engine instance operating within a confidential container to decrypt the re-encrypted chat message and initiate execution of a generative model to generate a response corresponding to the encrypted chat message marked with the private status.

16 . The non-transitory computer-readable storage medium of claim 15 , wherein the operations further comprise:

encrypting, using a client-controlled storage encryption key, response content for storage as encrypted chat history; and

storing the encrypted chat history on a server without access to the client-controlled storage encryption key such that plaintext of the response content is accessible to the generative AI system only during authorized processing within the trusted execution environment.

17 . The non-transitory computer-readable storage medium of claim 16 , wherein the operations further comprise performing a background processing operation using the generative AI system by:

decrypting, within the trusted execution environment and using the client-controlled storage encryption key, the encrypted chat history including the response content;

providing plaintext of the response content to the generative model to generate derived data associated with the response content; and

updating the encrypted chat history by re-encrypting, using the client-controlled storage encryption key, the response content and the derived data.

18 . The non-transitory computer-readable storage medium of claim 15 , wherein the operations further comprise:

executing, within the trusted execution environment, a safety classification operation to determine a safety flag for plaintext of the chat message or the response; and

based on the safety flag, applying one or more account-level restrictions to a user account associated with the client device.

19 . The non-transitory computer-readable storage medium of claim 15 , wherein the operations further comprise generating synthetic data by:

aggregating, within the trusted execution environment, plaintext from multiple chat messages satisfying a similarity criterion;

generating, using the generative model, a synthetic message that satisfies the similarity criterion with the multiple chat messages, while applying a differential privacy mechanism to limit contribution of individual chat messages from the multiple chat messages; and

outputting the synthetic message for use outside the trusted execution environment.

20 . The non-transitory computer-readable storage medium of claim 15 , wherein the operations further comprise:

determining that subsequent processing of a conversation should be performed outside of the trusted execution environment;

providing, for display via a graphical user interface of the client device, a notification indicating continuation of the conversation outside of the trusted execution environment; and

upon receiving consent from the client device, processing subsequent conversation operations outside of the trusted execution environment.