Mission-critical AI processor with record and replay support

1. A mission-critical processor, comprising:

an instruction buffer configured to store two or more instructions, wherein said two or more instructions comprise one or more artificial intelligence (AI) processing instructions and one or more hardware-initiated store instructions;

processing circuitry configured to compute an intermediate stage of an artificial intelligence (AI) model according to a current instruction if the current instruction belongs to said one or more AI processing instructions;

a data buffer configured to store data generated from executing the current instruction if the current instruction belongs to said one or more AI processing instructions; and

dedicated circuitry configured to transfer the data from the data buffer to a memory device of a computing system if the current instruction belongs to said one or more hardware-initiated store instructions, wherein the data is transferred without software involvement from a system processor of the computing system, and wherein the memory device is outside the mission-critical processor; and

wherein the dedicated circuitry is further configured to store computational states of the processing circuitry in a state buffer of the computing system, thereby allowing a second mission-critical processor of the computing system to resume operations associated with the computational states.

2. The mission-critical processor of claim 1 , wherein the data is transferred via a communication interface, and the communication interface is one of:

a peripheral component interconnect express (PCIe) interface; and

a network interface card (NIC).

3. The mission-critical processor of claim 1 , further comprising encryption circuitry configured to encrypt the data in the data buffer.

4. The mission-critical processor of claim 1 , wherein the processing circuitry includes a plurality of processing units comprising one or more of:

a dataflow processing unit (DPU) comprising a scalar computing unit and a vector computing unit; and

a tensor computing unit (TCU) comprising a cluster of DPUs, high-bandwidth memory devices, and input/output (I/O) devices.

5. The mission-critical processor of claim 4 , further comprising control circuitry configured to operate the processing circuitry in a low-power mode by turning off one or more processing units of the plurality of processing units.

6. The mission-critical processor of claim 1 , wherein the dedicated circuitry is further configured to store the data in a storage device of a remote computing device via a communication interface using a remote memory access protocol.

7. The mission-critical processor of claim 1 , wherein the instruction buffer is configured to store said two or more instructions by one of:

storing said two or more instructions prior to runtime; and

storing said one or more AI processing instructions prior to runtime and dynamically inserting said one or more hardware-initiated store instructions.

8. The mission-critical processor of claim 1 , wherein the dedicated circuitry, based on the hardware-initiated store instruction, is further configured to transfer contextual information associated with the mission-critical processor to the memory device, wherein the contextual information includes one or more of:

feature maps associated with the data in the data buffer;

information associated with one or more computational units of the mission-critical processor, wherein the information includes one or more of:

temperatures;

working conditions;

utilization; and

performance statistics.

9. A computer-implemented method for facilitating instructions to a mission-critical artificial intelligence (AI) processor, the method comprising:

fetching a current instruction from an instruction buffer, wherein the instruction buffer stores two more instructions comprising one or more artificial intelligence (AI) processing instructions and one or more hardware-initiated store instructions;

configuring processing circuitry to compute an intermediate stage of an AI model according to the current instruction if the current instruction belongs to said one or more AI processing instructions;

configuring a data buffer to store data generated from executing the current instruction if the current instruction belongs to said one or more AI processing instructions; and

configuring dedicated circuitry to transfer the data from the data buffer to a memory device of a computing system if the current instruction belongs to said one or more hardware-initiated store instructions, wherein the data is transferred without software involvement from a system processor of the computing system, and wherein the memory device is outside the mission-critical AI processor; and

wherein timing, frequency, or locations to insert a target hardware-initiated store instruction after a target AI processing instruction depends on a set of parameters and wherein the set of parameters includes available storage, communication bandwidth, and external inputs, which includes a policy of a datacenter, customer preferences, developer preferences, and environmental feedback.

10. The method of claim 9 , further comprising providing said two or more instructions to the mission-critical AI processor prior to runtime of the mission-critical AI processor.

11. A mission-critical system, the mission-critical system comprising:

a system processor;

a system memory device;

a communication interface; and

a first mission-critical processor coupled to the communication interface, wherein the first mission-critical processor comprises:

an instruction buffer configured to store two or more instructions, wherein said two or more instructions comprise one or more artificial intelligence (AI) processing instructions and one or more hardware-initiated store instructions;

processing circuitry configured to compute an intermediate stage of an artificial intelligence (AI) model according to a current instruction if the current instruction belongs to said one or more AI processing instructions;

a data buffer configured to store data generated from executing the current instruction if the current instruction belongs to said one or more AI processing instructions; and

dedicated circuitry configured to transfer the data from the data buffer to the system memory device of the mission-critical system without software involvement from the system processor of the mission-critical system, wherein the system memory device is outside the first mission-critical processor;

an operating module configured to load said two or more instructions in the instruction buffer of the first mission-critical processor,

a backup power source; and

power circuitry configured to:

detect a power failure of the mission-critical system;

switch the first mission-critical processor to the backup power source; and

reduce operations of the first mission-critical processor to save power.

12. The mission-critical system of claim 11 , wherein the communication interface is one of:

a peripheral component interconnect express (PCIe) interface; and

a network interface card (NIC).

13. The mission-critical system of claim 11 , wherein the processing circuitry of the first mission-critical processor includes a plurality of processing units comprising one or more of:

a dataflow processing unit (DPU) comprising a scalar computing unit and a vector computing unit; and

a tensor computing unit (TCU) comprising a cluster of DPUs, high-bandwidth memory devices, and input/output (I/O) devices.

14. The mission-critical system of claim 11 , further comprising a network interface configured to transfer the data from the system memory device to a remote computing system for replaying the data.

15. The mission-critical system of claim 11 , wherein the operating module is configured to load said two or more instructions by one of:

loading said two or more instructions prior to runtime; and

loading said one or more AI processing instructions prior to runtime and dynamically inserting said one or more AI processing instructions during runtime.

16. The mission-critical system of claim 11 , wherein the dedicated circuitry of the first mission-critical processor, based on the hardware-initiated store instruction, is further configured to transfer contextual information associated with the first mission-critical processor to the system memory device, wherein the contextual information includes one or more of:

feature maps associated with the data in the data buffer; and

information associated with one or more computational units of the first mission-critical processor, wherein the information includes one or more of:

temperatures;

working conditions;

utilization; and

performance statistics.

17. A mission-critical system, the mission-critical system comprising:

a system processor;

a system memory device;

a communication interface; and

a first mission-critical processor coupled to the communication interface, wherein the first mission-critical processor comprises:

an instruction buffer configured to store two or more instructions, wherein said two or more instructions comprise one or more artificial intelligence (AI) processing instructions and one or more hardware-initiated store instructions;

processing circuitry configured to compute an intermediate stage of an artificial intelligence (AI) model according to a current instruction if the current instruction belongs to said one or more AI processing instructions;

a data buffer configured to store data generated from executing the current instruction if the current instruction belongs to said one or more AI processing instructions; and

dedicated circuitry configured to transfer the data from the data buffer to the system memory device of the mission-critical system without software involvement from the system processor of the mission-critical system, wherein the system memory device is outside the first mission-critical processor;

an operating module configured to load said two or more instructions in the instruction buffer of the first mission-critical processor;

a state buffer configured to store computational states of the first mission-critical processor; and

high-availability circuitry configured to:

detect a failure of the first mission-critical processor;

load the computational states of the first mission-critical processor to a second mission-critical processor from the state buffer; and

resume operations associated with the computational states using the second mission-critical processor.