Invariants-based learning method and system for failure diagnosis in large scale computing systems
View Patent ↗A method system for diagnosing a detected failure in a computer system, compares a failure signature of the detected failure to an archived failure signature contained in a database to determine if the archived failure signature matches the failure signature of the detected failure. If the archived failure signature matches the failure signature of the detected failure, an archived solution is applied to the computer system that resolves the detected failure, the archived solution corresponding to a solution used to resolve a previously detected computer system failure corresponding to the archived failure signature in the database that matches the detected failure.
1. A method for diagnosing a detected failure in a computer system, the method comprising:
comparing, in a computer process, a failure signature of the detected failure to an archived failure signature contained in a database to determine if the archived failure signature matches the failure signature of the detected failure;
if the archived failure signature matches the failure signature of the detected failure, applying, in a computer process, an archived solution to the computer system that resolves the detected failure, the archived solution corresponding to a solution used to resolve a previously detected computer system failure corresponding to the archived failure signature in the database that matches the detected failure;
wherein the archived failure signature is based on a set of broken computer system invariants, the set of broken computer system invariants corresponding to the previously detected computer system failure;
further comprising constructing the database in a computer process prior to comparing the failure signature of the detected failure to the archived failure signature;
wherein the constructing the database includes extracting invariants from the computer system;
wherein the extracting the invariants includes:
modeling invariants of the computer system;
evaluating each of the invariants to determine whether it is broken;
counting the broken invariants to determine whether the number of the broken invariants meets a predetermined threshold number;
if the number of the broken invariants meets the predetermined threshold number deeming this result the previously detected computer system failure; and
combining the broken invariants into the set of broken invariants forming the archived failure signature of the previously detected computer system failure.
2. The method of claim 1 , wherein if the archived failure signature does not match the failure signature of the detected failure, further comprising resolving the detected failure using an alternative solution implemented manually or in a computer process.
3. The method of claim 2 , further comprising:
storing the alternative solution in the database or another database as another archived solution; and
storing the failure signature of the detected failure in the database as another archived failure signature.
4. The method of claim 3 , wherein the another archived solution is indexed to the another archived failure signature.
5. The method of claim 1 wherein the archived solution is contained in the database or another database.
6. The method of claim 5 , wherein the archived solution is indexed to the archived failure signature.
7. The method of claim 1 , wherein the failure signature of the detected failure is determined by extracting invariants from the computer system prior to the comparing of the failure signature of the detected failure to the archived failure signature contained in a database.
8. The method of claim 7 , wherein the extracting the invariants includes:
modeling invariants of the computer system;
evaluating each of the invariants to determine whether it is broken;
counting the broken invariants to determine whether the number of the broken invariants meets a predetermined threshold number;
if the number of the broken invariants meets the predetermined threshold number deeming this result the detected failure in the a computer system; and
combining the broken invariants into a set of broken computer system invariants, the set of broken invariants forming the failure signature of the detected failure in the computer system.
9. A system for diagnosing a detected failure in a computer system, the system comprising:
a database containing an archived failure signature; and
a processor associated with the database, the processor executing instructions for:
comparing a failure signature of the detected failure to the archived failure signature contained in the database to determine if the archived failure signature matches the failure signature of the detected failure;
if the archived failure signature matches the failure signature of the detected failure, applying an archived solution to the computer system that
resolves the detected failure, the archived solution corresponding to a solution used to resolve a previously detected computer system failure corresponding to the archived failure signature in the database that matches the detected failure;
wherein the processor executes further instructions for extracting invariants from the computer system prior to the comparing of the failure signature of the detected failure to the archived failure signature contained in the database, in order to determine the failure signature of the detected failure
wherein the extracting the invariants includes:
modeling invariants of the computer system;
evaluating each of the invariants to determine whether it is broken;
counting the broken invariants to determine whether the number of the broken invariants meets a predetermined threshold number;
if the number of the broken invariants meets the predetermined threshold number deeming this result the detected failure in the a computer system; and
combining the broken invariants into a set of broken computer system invariants, the set of broken invariants forming the failure signature of the detected failure in the computer system.
10. The system of claim 9 , wherein the archived failure signature is based on a set of broken computer system invariants, the set of broken computer system invariants corresponding to the previously detected computer system failure.
11. The system of claim 9 , wherein if the archived failure signature does not match the failure signature of the detected failure, the processor executing further instructions for resolving the detected failure using an alternative solution implemented manually or in a computer process.
12. The system of claim 11 , wherein the processor executes further instructions for:
storing the alternative solution in the database or another database as another archived solution; and
storing the failure signature of the detected failure in the database as another archived failure signature.
13. The system of claim 12 , wherein the another archived solution is indexed to the another archived failure signature.
14. The system of claim 9 , wherein the archived solution is contained in the database or another database.
15. The system of claim 14 , wherein the archived solution is indexed to the archived failure signature.
16. A method for diagnosing a detected failure in a computer system, the method comprising:
comparing, in a computer process, a failure signature of the detected failure to an archived failure signature contained in a database to determine if the archived failure signature matches the failure signature of the detected failure;
if the archived failure signature matches the failure signature of the detected failure, applying, in a computer process, an archived solution to the computer system that resolves the detected failure, the archived solution corresponding to a solution used to resolve a previously detected computer system failure corresponding to the archived failure signature in the database that matches the detected failure;
wherein the failure signature of the detected failure is determined by extracting invariants from the computer system prior to the comparing of the failure signature of the detected failure to the archived failure signature contained in a database;
wherein the extracting the invariants includes:
modeling invariants of the computer system;
evaluating each of the invariants to determine whether it is broken;
counting the broken invariants to determine whether the number of the broken invariants meets a predetermined threshold number;
if the number of the broken invariants meets the predetermined threshold number deeming this result the detected failure in the a computer system; and
combining the broken invariants into a set of broken computer system invariants, the set of broken invariants forming the failure signature of the detected failure in the computer system.