Cloud managed confidential workload error recovery and reporting
Confidential workload error recovery and reporting is disclosed. When an exception or other fault or error occurs in a trusted execution environment, context data is collected and stored in a remote access controller. The context data can be analyzed to determine a cause of the exception. The trusted execution environment is recovered based on the analysis of the context information and/or content of the trusted execution environment.
1 . A method comprising:
monitoring trusted execution environments present in one or more nodes of computing resources;
determining that an exception has occurred in a trusted execution environment operating on a node of the one or more nodes;
storing context information and confidential enclave content of the trusted execution environment in a memory specific to an out-of-band hardware remote access controller having a processor and an internal memory subsystem that is electrically and logically isolated from both the trusted execution environment and system memory of the node, and separate from the trusted execution environment and separate from memory of the one or more nodes;
transferring the context information and the confidential enclave content to the remote access controller;
deleting the context information and the confidential enclave content stored in the trusted execution environment responsive to confirming storage of the context information and the confidential enclave content in the isolated internal memory subsystem of the remote access controller;
recovering the trusted execution environment based on an analysis of the context information and/or the confidential enclave content to identify a cause of the exception and reconstructing the trusted execution environment on the node using the context information and the confidential enclave content stored in the remote access controller.
2 . The method of claim 1 , further comprising provisioning a workload in the trusted execution environment created at the node of the computing resources.
3 . The method of claim 1 , further comprising invoking an exception handler when the exception is determined to have occurred.
4 . The method of claim 3 , wherein the exception handler comprises the remote access controller.
5 . The method of claim 1 , further comprising generating a log entry in the remote access controller.
6 . The method of claim 5 , wherein the log entry includes the context information and/or the confidential enclave content.
7 . The method of claim 6 , wherein the log entry is configured to enable auditing of the exception.
8 . The method of claim 1 , further comprising generating an inference using a machine learning model trained on historical context information and root causes of exceptions.
9 . The method of claim 1 , wherein the context data includes register context of the trusted execution environment, data area store context, data regarding the exception, and telemetry data surrounding the exception, wherein the telemetry data includes data about the node and data about the trusted execution environment.
10 . The method of claim 1 , wherein recovering the trusted execution environment includes recovering the trusted execution to a point at which the exception occurred, rolling the trusted execution environment to a different point in time, or updating the trusted execution environment.
11 . A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
monitoring trusted execution environments present in one or more nodes of computing resources;
determining that an exception has occurred in a trusted execution environment operating on a node of the one or more nodes;
storing context information and confidential enclave content of the trusted execution environment in a memory specific to an out-of-band hardware remote access controller having a processor and an internal memory subsystem that is electrically and logically isolated from both the trusted execution environment and system memory of the node, and separate from the trusted execution environment and separate from memory of the one or more nodes;
transferring the context information and the confidential enclave content to the remote access controller;
deleting the context information and the confidential enclave content stored in the trusted execution environment responsive to confirming storage of the context information and the confidential enclave content in the isolated internal memory subsystem of the remote access controller;
recovering the trusted execution environment based on an analysis of the context information and/or the confidential enclave content to identify a cause of the exception and reconstructing the trusted execution environment on the node using the context information and the confidential enclave content stored in the remote access controller.
12 . The non-transitory storage medium of claim 11 , further comprising provisioning a workload in the trusted execution environment created at the node of the computing resources.
13 . The non-transitory storage medium of claim 11 , further comprising invoking an exception handler when the exception is determined to have occurred.
14 . The non-transitory storage medium of claim 13 , wherein the exception handler comprises the remote access controller.
15 . The non-transitory storage medium of claim 11 , further comprising generating a log entry in the remote access controller.
16 . The non-transitory storage medium of claim 15 , wherein the log entry includes the context information and/or the confidential enclave content.
17 . The non-transitory storage medium of claim 16 , wherein the log entry is configured to enable auditing of the exception.
18 . The non-transitory storage medium of claim 11 , further comprising generating an inference using a machine learning model trained on historical context information and root causes of exceptions.
19 . The non-transitory storage medium of claim 11 , wherein the context data includes register context of the trusted execution environment, data area store context, data regarding the exception, and telemetry data surrounding the exception, wherein the telemetry data includes data about the node and data about the trusted execution environment.
20 . The non-transitory storage medium of claim 11 , wherein recovering the trusted execution environment includes recovering the trusted execution to a point at which the exception occurred, rolling the trusted execution environment to a different point in time, or updating the trusted execution environment.