System and method to detect problematic code changes in firmware
An information handling system includes a storage and a baseboard management controller. The storage stores a current firmware release for the information handling system. The baseboard management controller generates a set of breakpoints for code differences from a previous firmware release to the current firmware release and executes multiple testcases for the current firmware release. Based on the execution of the testcases, the baseboard management controller stores breakpoint hitting events for the testcases in the storage. The baseboard management controller filters and ranks the breakpoint hitting events and maps the filtered and ranked breakpoint hitting events to the code differences. The baseboard management controller stores a reduced list of the code differences that are highly related to failed testcases of the current firmware release in the storage.
1 . An information handling system comprising:
a storage to store a current firmware release for the information handling system; and
a baseboard management controller to communicate with the storage, the baseboard management controller to:
generate a set of breakpoints for code differences from a previous firmware release to the current firmware release;
execute a plurality of testcases for the current firmware release;
based on the execution of the testcases, store breakpoint hitting events for the testcases in the storage;
filter and rank the breakpoint hitting events;
map the filtered and ranked breakpoint hitting events to the code differences; and
store, in the storage, a reduced list of the code differences that are highly related to failed testcases of the current firmware release.
2 . The information handling system of claim 1 , wherein the breakpoint hitting events being filtered and ranked includes the baseboard management controller further to: remove breakpoint hit events associated with background operations.
3 . The information handling system of claim 1 , wherein the breakpoint hitting events being filtered and ranked includes the baseboard management controller further to:
extract an ending timestamp for a failed testcase;
determine breakpoint hit events that occur before the ending timestamp; and
filter out breakpoint hit events that do not occur before the ending timestamp.
4 . The information handling system of claim 3 , wherein the breakpoint hitting events being filtered and ranked includes the baseboard management controller further to: rank the breakpoint hit events that occur before the ending timestamp.
5 . The information handling system of claim 4 , wherein the baseboard management controller utilizes term frequency-inverse document frequency to rank the breakpoint hit events that occur before the ending timestamp.
6 . The information handling system of claim 4 , wherein the breakpoint hitting events being filtered and ranked includes the baseboard management controller further to: filter out breakpoint hit events with a rank score below a threshold score.
7 . The information handling system of claim 1 , wherein the set of breakpoints for code differences are located in multiple threads of the current firmware release.
8 . The information handling system of claim 1 , wherein the baseboard management controller further to: associated different breakpoint hit events to different steps of the testcase.
9 . A method comprising:
storing, in a storage of an information handling system, a current firmware release for the information handling system;
generating, by a baseboard management controller of the information handling system, a set of breakpoints for code differences from a previous firmware release to the current firmware release;
executing a plurality of testcases for the current firmware release;
based on the execution of the testcases, storing breakpoint hitting events for the testcases stored in the storage;
filtering and ranking the breakpoint hitting events;
mapping, by the baseboard management controller, the filtered and ranked breakpoint hitting events to the code differences; and
storing, in the storage, a reduced list of the code differences that are highly related to failed testcases of the current firmware release.
10 . The method of claim 9 , wherein the filtering and ranking of the breakpoint hitting events includes the method further comprising: removing breakpoint hit events associated with background operations.
11 . The method of claim 9 , wherein the filtering and ranking of the breakpoint hitting events includes the method further comprising:
extracting an ending timestamp for a failed testcase;
determining breakpoint hit events that occur before the ending timestamp; and
filtering out breakpoint hit events that do not occur before the ending timestamp.
12 . The method of claim 11 , wherein the filtering and ranking of the breakpoint hitting events includes the method further comprising: ranking the breakpoint hit events that occur before the ending timestamp.
13 . The method of claim 12 , further comprising utilizing term frequency-inverse document frequency to rank the breakpoint hit events that occur before the ending timestamp.
14 . The method of claim 12 , wherein the filtering and ranking of the breakpoint hitting events includes the method further comprising filtering out breakpoint hit events with a rank score below a threshold score.
15 . The method of claim 9 , wherein the set of breakpoints for code differences are located in multiple threads of the current firmware release.
16 . The method of claim 9 , wherein the method further comprises: associating different breakpoint hit events to different steps of the testcase.
17 . An information handling system comprising:
a storage to store a current firmware release for the information handling system; and
a baseboard management controller to:
generate a set of breakpoints for code differences from a previous firmware release to the current firmware release;
execute a plurality of testcases for the current firmware release;
based on the execution of the testcases, store breakpoint hitting events for the testcases in the storage;
remove a first set of the breakpoint hit events associated with background operations;
extract an ending timestamp for a failed testcase;
determine a second set of the breakpoint hit events that occur before the ending timestamp;
filter out a third set of the breakpoint hit events that do not occur before the ending timestamp;
map the filtered out third set of the breakpoint hit events to the code differences; and
store, in the storage, a reduced list of the code differences that are highly related to the failed testcase of the current firmware release.
18 . The information handling system of claim 17 , wherein the baseboard management controller ranks a fourth set of the breakpoint hit events that occur before the ending timestamp.
19 . The information handling system of claim 18 , wherein the baseboard management controller utilizes term frequency-inverse document frequency to rank the second set of the breakpoint hit events that occur before the ending timestamp.
20 . The information handling system of claim 18 , wherein the baseboard management controller further to: filter out the fourth set of the breakpoint hit events with a rank score below a threshold score.