Software regression recovery via automated detection of problem change lists

1. A system for automatically recovering from software regression in a cloud computing environment, the system comprising:

a resolution server communicatively coupled to the cloud computing environment, the resolution server including an electronic processor configured to:

determine that a batch software update has been applied to the cloud computing environment;

in response to determining that a batch software update has been applied, transmit a problem request to an event listener server;

receive, from the event listener server, a problem statement including a stack trace, the stack trace referencing a code segment;

determine, based on the code segment referenced by the stack trace, a software feature indicator;

transmit the software feature indicator to a root cause analyzer;

receive, from the root cause analyzer, a change list indicator and a relevancy score associated with the change list indicator; and

when the relevancy score exceeds a relevancy threshold, perform a mitigation action based on the change list indicator.

2. The system of claim 1 , wherein the electronic processor is further configured to

in response to determining that the batch software update has been applied, initialize a batch update timer; and

transmit the problem request to the event listener server when the batch update timer exceeds an update timer threshold.

3. The system of claim 1 , wherein the electronic processor is further configured to determine that a batch software update has been applied by receiving, from a software updater, a notification indicating that the software updater has pushed a batch software update to the cloud computing environment.

4. The system of claim 3 , wherein the electronic processor is further configured to

receive, from the software updater, metadata associated with the batch software update, the metadata including a plurality of key value pairs, each including a software feature and a code segment associated with the software feature; and

determine the software feature indicator based on the stack trace and the plurality of key value pairs.

5. The system of claim 1 , wherein the electronic processor is further configured to

receive, from the software updater, metadata associated with the batch software update;

determine, based on the metadata, whether a change list reversion mechanism exists for a change list identified by the change list indicator; and

in response to determining that a change list reversion mechanism does not exist for the change list, perform the mitigation action by transmitting an electronic message to a change list owner of the change list.

6. The system of claim 5 , wherein the electronic processor is further configured to, in response to determining that a change list reversion mechanism exists for the change list, perform the mitigation action by executing the change list reversion mechanism.

7. The system of claim 6 , wherein the change list reversion mechanism is one selected from the group consisting of a flight on rails service and a killswitch service.

8. The system of claim 1 , wherein the problem statement is based on a failure threshold.

9. A method for automatically recovering from software regression in a cloud computing environment, the method comprising:

determining, with an electronic processor, that a batch software update has been applied to the cloud computing environment;

in response to determining that a batch software update has been applied, transmitting a problem request to an event listener server;

receiving, from the event listener server, a problem statement including a stack trace, the stack trace referencing a code segment;

determining, based on the code segment referenced by the stack trace, a software feature indicator;

transmitting the software feature indicator to a root cause analyzer;

receiving, from the root cause analyzer, a change list indicator and a relevancy score associated with the change list indicator; and

performing a mitigation action based on the change list indicator when the relevancy score exceeds a relevancy threshold.

10. The method of claim 9 , further comprising;

in response to determining that the batch software update has been applied, initializing a batch update timer; and

transmitting the problem request to the event listener server when the batch update timer exceeds an update timer threshold.

11. The method of claim 9 , wherein determining that a batch software update has been applied includes receiving, from a software updater, a notification indicating that the software updater has pushed a batch software update to the cloud computing environment.

12. The method of claim 11 , further comprising:

receiving, from the software updater, metadata associated with the batch software update, the metadata including a plurality of key value pairs, each including a software feature and a code segment associated with the software feature; and

determining the software feature indicator based on the stack trace and the plurality of key value pairs.

13. The method of claim 9 , further comprising:

receiving, from the software updater, metadata associated with the batch software update;

determining, based on the metadata, whether a change list reversion mechanism exists for a change list identified by the change list indicator; and

in response to determining that a change list reversion mechanism does not exist for the change list, perform the mitigation action by transmitting an electronic message to a change list owner of the change list.

14. The method of claim 13 , further comprising:

in response to determining that a change list reversion mechanism exists for the offending change list, perform the mitigation action by executing the change list reversion mechanism and transmitting an electronic message to a change list owner of the change list indicating that the change list was reverted.

15. The method of claim 14 , executing a change list reversion mechanism includes performing one selected from the group consisting of activating a flight on rails service and setting a killswitch service flag.

16. The method of claim 9 , wherein the problem statement is based on a failure threshold.

17. The method of claim 9 , further comprising:

determining the relevancy score using a machine learning analysis of batch software update metadata.

18. A non-transitory computer-readable medium including instructions executable by an electronic processor to perform a set of functions, the set of functions comprising:

receiving, from a software updater, a notification indicating that the software updater has pushed a batch software update to the cloud computing environment and metadata associated with the batch software update;

in response to receiving the notification, transmitting a problem request to an event listener server;

receiving, from the event listener server, a problem statement including a stack trace, the stack trace referencing a code segment;

determining, based on the code segment referenced in the stack trace, a software feature indicator;

transmitting the software feature indicator to a root cause analyzer;

receiving, from the root cause analyzer, a change list indicator and a relevancy score associated with the change list indicator;

when the relevancy score exceeds a relevancy threshold, determining, based on the metadata, whether a change list reversion mechanism exists for a change list identified by the change list indicator; and

in response to determining that a change list reversion mechanism exists for the change list, executing the change list reversion mechanism.

19. The non-transitory computer-readable medium of claim 18 , the set of functions further comprising:

in response to determining that a change list reversion mechanism does not exist for the change list, transmitting an electronic message to a change list owner of the change list.

20. The non-transitory computer-readable medium of claim 18 , the set of functions further comprising:

in response to receiving the notification that the batch software update has been applied, initializing a batch update timer; and

transmitting the problem request to the event listener server when the batch update timer exceeds an update timer threshold.