IP Library Granted Patent US 11,994,940
Granted Patent B2
US 11,994,940 · App. 17/384,361 · Granted May 28, 2024

Fault processing method, related device, and computer storage medium

Inventors: Wei Liu (Shenzhen, CN); Feiyang Gu (Dongguan, CN); Shen Wang (Shenzhen, CN); Mohua Li (Beijing, CN)
Assignee: Huawei Cloud Computing Technologies Co., Ltd.
G06F11/0793G06F11/0706G06F11/0751G06F13/24G06F13/4282G06F2213/0026
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,994,940
App. No.
17/384,361
Granted
May 28, 2024
Kind
B2
Abstract

A fault processing method includes: a fault processing apparatus receives first hardware fault information generated by a first PCIe device on a first PCIe link. The first hardware fault information includes a device identifier of the first PCIe device and is used to indicate that a hardware fault occurs on the first PCIe device. Further, the fault processing apparatus performs, based on the first hardware fault information, fault recovery on the first PCIe link on which the first PCIe device is located, and interrupts a software service related to the first PCIe device.

Claims (81)

1. A fault processing method comprising:

receiving, by a fault processing apparatus, first hardware fault information generated by a first peripheral component interconnect express (PCIe) device on a first high-speed serial computer expansion bus standard PCIe link,

wherein the first hardware fault information indicates that a hardware fault has occurred on the first PCIe device, and

wherein the first hardware fault information comprises a device identifier of the first PCIe device;

distinguishing, by the fault processing apparatus, between whether the first PCIe device is a parallel processor, whether the first PCIe device is a network adapter, and whether the first PCIe device is another device,

wherein determining the first PCIe device is a network adapter is based on a fault recovery function being pre-registered with the first PCIe device; and

performing, by the fault processing apparatus based on the first hardware fault information and based on whether the first PCIe device is a parallel processor, a network adapter, or another device, fault recovery on the first PCIe link on which the first PCIe device is located, and interrupting a software service related to the first PCIe device.

2. The method according to claim 1 , wherein the software service related to the first PCIe device is interrupted without interrupting a software service related to a second PCIe device, wherein the second PCIe device is located on a second PCIe link different from the first PCIe link.

3. The method according to claim 1 , wherein after performing fault recovery on the first PCIe link and interrupting the software service related to the first PCIe device, the method further comprises:

obtaining, by the fault processing apparatus, hardware fault recovery success information that carries the device identifier of the first PCIe device; and

recovering, by the fault processing apparatus, the software service related to the first PCIe device based on the fault recovery success information.

4. The method according to claim 3 , wherein the first PCIe device is determined to be a parallel processor based on the device identifier of the first PCIe device that is carried in the first hardware fault information.

5. The method according to claim 4 , wherein the fault processing apparatus records a correspondence between the device identifier of the first PCIe device and a virtual machine identifier of a virtual machine that uses the first PCIe device, and wherein interrupting the software service related to the first PCIe device comprises:

determining, by the fault processing apparatus based on the correspondence and the device identifier of the first PCIe device, the virtual machine identifier of the virtual machine that uses the first PCIe device; and

sending, by the fault processing apparatus based on the virtual machine identifier, a notification message to the virtual machine that uses the first PCIe device, wherein the notification message indicates that the virtual machine is to stop running a data processing service related to the first PCIe device.

6. The method according to claim 4 , wherein recovering the software service related to the first PCIe device based on the fault recovery success information comprises:

determining, by the fault processing apparatus based on the device identifier of the first PCIe device that is carried in the fault recovery success information, a virtual machine identifier of a virtual machine that uses the first PCIe device; and

sending, by the fault processing apparatus based on the virtual machine identifier, a notification message to the virtual machine that uses the first PCIe device, wherein the notification message indicates that the virtual machine that uses the first PCIe device is to restart to recover a data processing service related to the first PCIe device.

7. The method according to claim 4 , wherein before receiving the first hardware fault information, the method further comprises:

recording, by the fault processing apparatus, status information stored in a register of each PCIe device on the first PCIe link; and

wherein performing fault recovery on the first PCIe link comprises:

disabling, by the fault processing apparatus, a fault reporting function of each PCIe device on the first PCIe link, to stop generating a hardware fault interrupt signal;

resetting, by the fault processing apparatus, a downstream port of each PCIe device on the first PCIe link, so that the register of each PCIe device on the first PCIe link is recovered to an initial state;

writing, by the fault processing apparatus, the status information into the register of each PCIe device on the first PCIe link; and

enabling, by the fault processing apparatus, the fault reporting function of each PCIe device on the first PCIe link.

8. The method according to claim 3 , wherein the first PCIe device is determined to be a network adapter based on the device identifier of the first PCIe device that is carried in the first hardware fault information.

9. The method according to claim 8 , wherein the fault processing apparatus records a correspondence between the device identifier of the first PCIe device and a virtual machine identifier of a virtual machine that uses the first PCIe device, and wherein interrupting the software service related to the first PCIe device comprises:

determining, by the fault processing apparatus based on the correspondence and the device identifier of the first PCIe device, the virtual machine identifier of the virtual machine that uses the first PCIe device; and

sending, by the fault processing apparatus, a notification message based on the virtual machine identifier, wherein the notification message indicates that the virtual machine is to stop running a data exchange service related to the first PCIe device.

10. The method according to claim 8 , wherein recovering the software service related to the first PCIe device based on the fault recovery success information comprises:

determining, by the fault processing apparatus based on the device identifier of the first PCIe device that is carried in the fault recovery success information, a virtual machine identifier of a virtual machine that uses the first PCIe device; and

sending, by the fault processing apparatus, a notification message to the virtual machine based on the virtual machine identifier, wherein the notification message indicates that the virtual machine is to restart to recover the data exchange service related to the first PCIe device.

11. A computing device comprising:

a processor;

a memory;

a communications interface; and

a bus;

wherein the processor, the communications interface, and the memory communicate with each other through the bus;

wherein the communications interface is configured to receive and send data;

wherein the memory is configured to store processor-executable instructions; and

wherein the processor is configured to execute the processor-executable instructions to facilitate the following being performed by the computing device:

receiving first hardware fault information generated by a first peripheral component interconnect express (PCIe) device on a first high-speed serial computer expansion bus standard PCIe link, wherein the first hardware fault information indicates that a hardware fault has occurred on the first PCIe device, and the first hardware fault information comprises a device identifier of the first PCIe device;

distinguishing between whether the first PCIe device is a parallel processor, whether the first PCIe device is a network adapter, and whether the first PCIe device is another device,

wherein determining the first PCIe device is a network adapter is based on a fault recovery function being pre-registered with the first PCIe device; and

performing, based on the first hardware fault information and based on whether the first PCIe device is a parallel processor, a network adapter, or another device, fault recovery on the first PCIe link on which the first PCIe device is located, and interrupting a software service related to the first PCIe device.

12. The computing device according to claim 11 , wherein the processor is further configured to execute the processor-executable instructions to facilitate the following being performed by the computing device:

interrupting a software service related to the first PCIe device without interrupting a software service related to a second PCIe device, wherein the second PCIe device is located on a second PCIe link that is different from the first PCIe link.

13. The computing device according to claim 11 , wherein the processor is further configured to execute the processor-executable instructions to facilitate the following being performed by the computing device:

obtaining hardware fault recovery success information that carries the device identifier of the first PCIe device; and

recovering the software service related to the first PCIe device based on the fault recovery success information.

14. The computing device according to claim 13 , wherein the first PCIe device is determined to be a parallel processor based on the device identifier of the first PCIe device that is carried in the first hardware fault information.

15. The computing device according to claim 14 , wherein interrupting the software service related to the first PCIe device comprises:

determining, based on a correspondence between the device identifier of the first PCIe device and a virtual machine identifier of a virtual machine that uses the first PCIe device, the virtual machine identifier of the virtual machine that uses the first PCIe device; and

sending, based on the virtual machine identifier, a notification message to the virtual machine that uses the first PCIe device, wherein the notification message indicates that the virtual machine is to stop running a data processing service related to the first PCIe device.

16. The computing device according to claim 14 , wherein recovering the software service related to the first PCIe device based on the fault recovery success information comprises:

determining, based on the device identifier of the first PCIe device that is carried in the fault recovery success information, a virtual machine identifier of the virtual machine that uses the first PCIe device; and

sending, based on the virtual machine identifier, a notification message to the virtual machine that uses the first PCIe device, wherein the notification message indicates that the virtual machine is to restart to recover the data processing service related to the first PCIe device.

17. The computing device according to claim 14 , wherein the processor is further configured to execute the processor-executable instructions to facilitate the following being performed by the computing device:

recording status information stored in a register of each PCIe device on the first PCIe link; and

wherein performing fault recovery on the first PCIe link comprises:

disabling a fault reporting function of each PCIe device on the first PCIe link, to stop generating a hardware fault interrupt signal;

resetting a downstream port of each PCIe device on the first PCIe link, so that the register of each PCIe device on the first PCIe link is recovered to an initial state;

writing the status information into the register of each PCIe device on the first PCIe link; and

enabling the fault reporting function of each PCIe device on the first PCIe link.

18. The computing device according to claim 13 , wherein the first PCIe device is determined to be a network adapter based on the device identifier of the first PCIe device that is carried in the first hardware fault information.

19. The computing device according to claim 18 , wherein interrupting the software service related to the first PCIe device comprises:

determining, based on a correspondence between the device identifier of the first PCIe device and a virtual machine identifier of a virtual machine that uses the first PCIe device, the virtual machine that uses the PCIe device; and

sending a notification message based on the virtual machine identifier, wherein the notification message indicates that the virtual machine is to stop running a data exchange service related to the first PCIe device.

20. The computing device according to claim 18 , wherein recovering the software service related to the first PCIe device based on the fault recovery success information comprises:

determining, based on a correspondence between the device identifier of the first PCIe device and a virtual machine identifier of a virtual machine that uses the first PCIe device, the virtual machine identifier of the virtual machine that uses the first PCIe device; and

sending a notification message to the virtual machine based on the virtual machine identifier, wherein the notification message indicates that the virtual machine is to restart to recover the data exchange service related to the first PCIe device.

21. A non-transitory computer readable medium configured to store processor-executable instructions that, when executed by a processor, facilitate the processor to provide at least the following operations:

receive first hardware fault information generated by a first peripheral component interconnect express (PCIe) device on a first high-speed serial computer expansion bus standard PCIe link,

wherein the first hardware fault information indicates that a hardware fault has occurred on the first PCIe device, and

wherein the first hardware fault information comprises a device identifier of the first PCIe device;

distinguish between whether the first PCIe device is a parallel processor, whether the first PCIe device is a network adapter, and whether the first PCIe device is another device,

wherein determining the first PCIe device is a network adapter is based on a fault recovery function being pre-registered with the first PCIe device;

perform, based on the first hardware fault information and based on whether the first PCIe device is a parallel processor, a network adapter, or another device, fault recovery on the first PCIe link on which the first PCIe device is located; and

interrupt a software service related to the first PCIe device.

22. The non-transitory computer readable medium according to claim 21 , wherein the stored processor-executable instructions include further processor-executable instructions to facilitate the following operations by the processor:

interrupting the software service related to the first PCIe device without interrupting a software service related to a second PCIe device, wherein the second PCIe device is located on a second PCIe link that is different from the first PCIe link.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 1, 2024
From: HUAWEI TECHNOLOGIES CO., LTD.
To: HUAWEI CLOUD COMPUTING TECHNOLOGIES CO., LTD.
Reel/Frame 067281/0269 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2024
From: LIU, WEI; GU, FEIYANG; WANG, SHEN; LI, MOHUA
To: HUAWEI TECHNOLOGIES CO., LTD.
Reel/Frame 066827/0517 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2022
From: HUAWEI TECHNOLOGIES CO., LTD.
To: HUAWEI CLOUD COMPUTING TECHNOLOGIES CO., LTD.
Reel/Frame 059267/0088 →
Cited By (2)
US 12,254,322 US 12,373,273