Information handling system pre-boot fault management
An information handling system pre-boot fault monitor tracks errors detected before boot of an operating system and stores the errors in persistent memory as error hashes generated from information associated with the error. Corrective actions associated with error hashes are determined by data mining error hashes provided from a population of deployed systems and stored in the persistent memory of the deployed systems. As the pre-boot fault monitor detects errors, a matching comparison between detected error hashes and stored corrective action hashes provides pre-boot instructions with corrective actions so that boot can be completed and the error managed with the operating system after POST.
1. An information handling system comprising:
a housing;
plural processing components disposed in the housing and operable to execute instructions to process information, the processing components including at least a processor, the instructions including an operating system executed on the processor;
a memory disposed in the housing and interfaced with the processor, the memory operable to store the information;
an embedded controller having pre-boot instructions operable to power on the processing components and initiate execution of the operating system on the processor;
non-volatile memory interfaced with the pre-boot instructions and operable to persistently store information written by the processing components;
a pre-boot fault monitor executing on the processing components and operable to detect errors, the pre-boot fault monitor having error handling for known errors having corrective actions and general error handling for unanticipated errors that lack corrective actions, the general error handling applying a predefined set of error information including an operating state at the time of the detected errors to generate a hash and store the hash in the non-volatile memory; and
a fault table stored in non-volatile memory accessible by the pre-boot fault monitor, the fault table storing hashes of errors associated with corrective actions, the pre-boot fault monitor comparing the hash generated by a detected error with the fault table and applying corrective actions for detected hashes that match fault table hashes.
2. The information handling system of claim 1 wherein the hash comprises at least an identifier of a module of the pre-boot instructions and an offset of the module, offset associated with detection of an error.
3. The information handling system of claim 1 wherein the hash comprises at least an identifier of a hardware processing component disposed in the housing.
4. The information handling system of claim 1 wherein the non-volatile memory stores hashes generated for detected errors in a circular buffer.
5. The information handling system of claim 1 wherein the pre-boot fault monitor is further operable to generate a non-error hash before initialization of one or more hardware processing components.
6. The information handling system of claim 1 further comprising a fault database disposed at a network location, the fault database storing error hashes detected at plural information handling systems and communicated by the network.
7. The information handling system of claim 6 further comprising fault analytics interfaced with the fault database, the fault analytics comparing error hashes across plural hardware platforms to associate errors with varying combinations of hardware processing components and boot instruction module offsets.
8. The information handling system of claim 1 wherein the pre-boot fault monitor reads hashes in the fault table stored from a most recent previous boot before initialization of hardware components.
9. A method for monitoring faults at an information handling system, the method comprising:
detecting application of power at an embedded controller of the information handling system;
in response to detecting application of power, executing pre-boot instructions stored in persistent memory, the pre-boot instructions resulting in boot of an operating system;
monitoring the executing of pre-boot instructions to detect errors;
in response to detecting an error, generating an error hash by applying a predefined set of error information related to operating conditions of the information handling system at a time of the error; and
storing the error hash in non-volatile memory accessible by the pre-boot instructions.
10. The method of claim 9 further comprising:
detecting upon the application of power that an error hash is stored in the non-volatile memory; and
in response to detecting the error hash, reading the error hash before initiating the pre-boot instructions.
11. The method of claim 10 further comprising:
in response to detecting the error hash, comparing the error hash with error hashes stored in a fault table;
finding a match between the error hash and an error hash stored in the fault table; and
applying a corrective action associated with the error hash in the fault table.
12. The method of claim 9 wherein the error hash comprises an identifier of a module of the pre-boot instructions and an offset in the module at which the error occurred.
13. The method of claim 9 wherein the error hash comprises an identifier of a hardware component associated with the detected error.
14. The method of claim 9 further comprising:
writing an error hash before initialization of one or more hardware components, the error hash identifying the component and initialization; and
deleting the error hash after completion of initialization without an error.
15. A system for managing information handling system faults, the system comprising:
a pre-boot fault monitor integrated in plural information handling systems, the pre-boot fault manager applying predefined operating state information of an information handling system at detection of an error to generate a hash and storing the hash in the non-volatile memory;
a fault database interfaced with the pre-boot fault manager through a network and operable to store hashes provided from the plural information handling systems;
a fault analytics engine interfaced with the fault database and operable to associate one or more of the hashes with a fault and a corrective action; and
a fault table stored in persistent memory of each information handling system and accessible by the pre-fault monitor, the fault analytics engine updating the fault table with hashes and corrective actions through the network, the pre-fault monitor comparing detected error hashes with fault table hashes to determine corrective actions.
16. The system of claim 15 wherein the hash comprises an identifier of the portion of code executing to pre-boot the information handling system at the detection of the error.
17. The system of claim 15 wherein the hash comprises an identifier of a hardware component interacting with pre-boot instructions of the information handling system at the detection of the error.
18. The system of claim 15 wherein the hash comprises a communication bus state at detection of the error.
19. The system of claim 15 wherein the hash comprises a chipset status having an HECI status, USB state, PCI state and power management controller cause of last reset.
20. The system of claim 15 wherein the hash comprises an IP and call stack state.