Dynamically detecting and interrupting excessive execution time

1. A method, comprising:

executing, by a first function of a first process executing on a processor, a plurality of calls to a second function of a second process;

programmatically generating, based on a respective amount of time required for each of the plurality of calls to complete, a time threshold for calls from the first function to the second function; and

subsequent to the plurality of calls completing:

storing, by an operating system (OS) kernel executing on the processor and in a queue of the OS kernel, an indication that the first function of the first process executing on the processor has made an additional call to the second function of the second process;

collecting process data for at least one of the first process and the second process;

determining, by the OS kernel, that an amount of time that has elapsed since the first function of the first process made the additional call to the second function of the second process exceeds the programmatically defined time threshold;

storing the queue and the process data as part of a failure data capture; and

performing a predefined operation on at least one of the first process and the second process.

2. The method of claim 1 , wherein the process data comprises: (i) contents of a respective stack of each process, (ii) a respective use of the processor by each process, (iii) a respective use of a memory of each process, and (iv) a total amount of use of the processor.

3. The method of claim 2 ,

wherein the time threshold is programmatically generated based on: (i) an average amount of time required for the plurality of calls to complete, (ii) a minimum amount of time required for one of the plurality of calls to complete, and (iii) a maximum amount of time required for one of the plurality of calls to complete, and wherein the time threshold is defined upon determining that the maximum amount of time required for one of the plurality of calls to complete is greater than a minimum time threshold for performing the predefined operation.

4. The method of claim 3 , further comprising:

upon determining that the first function in the first process calls the second function in the second process, starting, by the OS kernel, a timer used to determine that the time threshold has been exceeded by the elapsed time.

5. The method of claim 4 , further comprising, prior to determining that the amount of time exceeds the time threshold:

determining, by the OS kernel, that the second function in the second process calls a third function in a third process; and

storing, in the queue, an indication that the second function called the third function.

6. The method of claim 5 , further comprising:

upon determining the call to the third function in the third process completes, removing, from the queue, the indication that the second function in the second process called the third function in the third process.

7. The method of claim 5 , wherein the first process accesses a first address space in a memory, wherein the second process addresses a second address space in the memory, different than the first address space in the memory, wherein the queue is stored in a protected area of the memory accessible by the OS kernel, wherein a content of the memory is stored as part of the failure data capture, wherein the predefined operation comprises one of: (i) terminating, (ii) interrupting, and (iii) restarting at least one of the first process and the second process, wherein the at least one of the first process and the second process is selected based on: (i) an execution time of the first process and the second process, (ii) a priority of the first process, and (iii) a priority of the second process.

8. A system, comprising:

a computer processor; and

a memory containing a program which when executed by the processor performs an operation comprising:

executing, by a first function of a first process executing on the processor, a plurality of calls to a second function of a second process;

programmatically generating, based on a respective amount of time required for each of the plurality of calls to complete, a time threshold for calls from the first function to the second function; and

subsequent to the plurality of calls completing:

storing, by an operating system (OS) kernel executing on the processor and in a queue of the OS kernel, an indication that the first function of the first process executing on the processor has made an additional call to the second function of the second process;

collecting process data for at least one of the first process and the second process;

determining, by the OS kernel, that an amount of time that has elapsed since the first function of the first process made the additional call to the second function of the second process exceeds the programmatically defined time threshold;

storing the queue and the process data as part of a failure data capture; and

performing a predefined operation on at least one of the first process and the second process.

9. The system of claim 8 , wherein the process data comprises: (i) contents of a respective stack of each process, (ii) a respective use of the processor by each process, (iii) a respective use of a memory of each process, and (iv) a total amount of use of the processor.

10. The system of claim 9 ,

wherein the time threshold is programmatically generated based on: (i) an average amount of time required for the plurality of calls to complete, (ii) a minimum amount of time required for one of the plurality of calls to complete, and (iii) a maximum amount of time required for one of the plurality of calls to complete, and wherein the time threshold is defined upon determining that the maximum amount of time required for one of the plurality of calls to complete is greater than a minimum time threshold for performing the predefined operation.

11. The system of claim 10 , the operation further comprising:

upon determining that the first function in the first process calls the second function in the second process, starting, by the OS kernel, a timer used to determine that the time threshold has been exceeded by the elapsed time.

12. The system of claim 11 , the operation further comprising, prior to determining that the amount of time exceeds the time threshold:

determining, by the OS kernel, that the second function in the second process calls a third function in a third process; and

storing, in the queue, an indication that the second function called the third function.

13. The system of claim 12 , the operation further comprising:

upon determining the call to the third function in the third process completes, removing, from the queue, the indication that the second function in the second process called the third function in the third process.

14. The system of claim 13 , wherein the first process accesses a first address space in a memory, wherein the second process addresses a second address space in the memory, different than the first address space in the memory, wherein the queue is stored in a protected area of the memory accessible by the OS kernel, wherein a content of the memory is stored as part of the failure data capture, wherein the predefined operation comprises one of: (i) terminating, (ii) interrupting, and (iii) restarting at least one of the first process and the second process, wherein the at least one of the first process and the second process is selected based on: (i) an execution time of the first process and the second process, (ii) a priority of the first process, and (iii) a priority of the second process.

15. A computer program product comprising:

a non-transitory computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors including a first processor to perform an operation comprising:

executing, by a first function of a first process executing on the processor, a plurality of calls to a second function of a second process;

programmatically generating, based on a respective amount of time required for each of the plurality of calls to complete, a time threshold for calls from the first function to the second function; and

subsequent to the plurality of calls completing:

storing, by an operating system (OS) kernel executing on the processor and in a queue of the OS kernel, an indication that the first function of the first process executing on the processor has made an additional call to the second function of the second process;

collecting process data for at least one of the first process and the second process;

determining, by the OS kernel, that an amount of time that has elapsed since the first function of the first process made the additional call to the second function of the second process exceeds the programmatically defined time threshold;

storing the queue and the process data as part of a failure data capture; and

performing a predefined operation on at least one of the first process and the second process.

16. The computer program product of claim 15 , wherein the process data comprises: (i) contents of a respective stack of each process, (ii) a respective use of the processor by each process, (iii) a respective use of a memory of each process, and (iv) a total amount of use of the processor.

17. The computer program product of claim 16 ,

wherein the time threshold is programmatically generated based on: (i) an average amount of time required for the plurality of calls to complete, (ii) a minimum amount of time required for one of the plurality of calls to complete, and (iii) a maximum amount of time required for one of the plurality of calls to complete, and wherein the time threshold is defined upon determining that the maximum amount of time required for one of the plurality of calls to complete is greater than a minimum time threshold for performing the predefined operation.

18. The computer program product of claim 17 , the operation further comprising:

upon determining that the first function in the first process calls the second function in the second process, starting, by the OS kernel, a timer used to determine that the time threshold has been exceeded by the elapsed time.

19. The computer program product of claim 18 , the operation further comprising, prior to determining that the amount of time exceeds the time threshold:

determining, by the OS kernel, that the second function in the second process calls a third function in a third process; and

storing, in the queue, an indication that the second function called the third function.

20. The computer program product of claim 19 , wherein the first process accesses a first address space in a memory, wherein the second process addresses a second address space in the memory, different than the first address space in the memory, wherein the queue is stored in a protected area of the memory accessible by the OS kernel, wherein a content of the memory is stored as part of the failure data capture, wherein the predefined operation comprises one of: (i) terminating, (ii) interrupting, and (iii) restarting at least one of the first process and the second process, wherein the at least one of the first process and the second process is selected based on: (i) an execution time of the first process and the second process, (ii) a priority of the first process, and (iii) a priority of the second process, the operation further comprising:

upon determining the call to the third function in the third process completes, removing, from the queue, the indication that the second function in the second process called the third function in the third process.