Fault tolerant retry subsystem and method
View Patent ↗A fault-tolerant retry subsystem, method and computer program product are provided in order to automatically retry to deliver a message. In the context of a fault-tolerant retry subsystem, processing circuitry is provided to receive notification that delivery of a message to a subscriber has failed. The processing circuitry also defines a time interval at which delivery of the message is to be retried. The message is to be subjected to a plurality of retry attempts in an instance in which the delivery of the message to the subscriber continues to fail. The time interval is defined in a manner that depends upon which of the plurality of retry attempts is to be performed. As such, the time interval varies between at least some of the retry attempts. The processing circuitry causes the delivery of the message to be retried following expiration the time interval.
1. A fault-tolerant retry subsystem for automatically retrying to deliver a message, the fault-tolerant retry subsystem comprising a processing circuitry, wherein the processing circuitry comprises a processor and a primary memory accessed by the processor during execution of functions of the fault-tolerant retry subsystem, wherein the processing circuitry is configured to:
receive notification that delivery of the message to a subscriber has failed;
define a time interval at which delivery of the message is to be retried, wherein the message is to be subjected to a plurality of retry attempts in an instance in which the delivery of the message to the subscriber continues to fail, and wherein the time interval is defined in a manner that is dependent upon which of the plurality of retry attempts is to be performed such that the time interval varies between at least some of the retry attempts;
cause a retry attempt to be scheduled in a secondary persistent storage in response to receipt of the notification that the delivery of the message has failed and following definition of the time interval, wherein the secondary persistent storage is slower to respond than the primary memory;
following scheduling of the retry attempt in the secondary persistent storage and in response to notification of the retry attempt by the secondary persistent storage following expiration of the time interval, reinstantiate the retry attempt in the primary memory; and
cause the delivery of the message to be retried following expiration of the time interval and reinstantiation of the retry attempt.
2. A fault-tolerant retry subsystem according to claim 1 wherein the processing circuitry is configured to define the time interval by defining the time interval such that the time interval progressively increases as the message is subjected to additional retry attempts.
3. A fault-tolerant retry subsystem according to claim 2 wherein the time interval progressively increases in a non-linear manner as the message is subjected to additional retry attempts.
4. A fault-tolerant retry subsystem according to claim 1 wherein the processing circuitry is configured to receive publication of an event associated with the retry attempt following expiration of the time interval, wherein the retry attempt is reinstantiated in the primary memory in response to receipt of publication of the event.
5. A fault-tolerant retry subsystem according to claim 1 wherein the processing circuitry is configured to reinstantiate the retry attempt in an asynchronous manner.
6. A fault-tolerant retry subsystem according to claim 1 wherein the processing circuitry is further configured to:
maintain a count of retry attempts of a respective message; and
halt further retry attempts in response to the count of retry attempts satisfying a predefined limit.
7. A fault-tolerant retry subsystem according to claim 1 wherein the secondary persistent storage is implemented by a horizontally scaled set of machines.
8. A method of a fault-tolerant retry subsystem for automatically retrying to deliver a message, wherein the fault-tolerant retry subsystem comprises a processor and a primary memory accessed by the processor during execution of functions of the fault-tolerant retry subsystem, and wherein the method comprises:
receiving notification that delivery of the message to a subscriber has failed;
defining a time interval at which delivery of the message is to be retried, wherein the message is to be subjected to a plurality of retry attempts in an instance in which the delivery of the message to the subscriber continues to fail, and wherein defining the time interval comprises defining the time interval in a manner that is dependent upon which of the plurality of retry attempts is to be performed such that the time interval varies between at least some of the retry attempts;
causing a retry attempt to be scheduled in a secondary persistent storage in response to receipt of the notification that the delivery of the message has failed and following definition of the time interval, wherein the secondary persistent storage is slower to respond than the primary memory;
following scheduling of the retry attempt in the secondary persistent storage and in response to notification of the retry attempt by the secondary persistent storage following expiration of the time interval, reinstantiating the retry attempt in the primary memory; and
causing the delivery of the message to be retried following expiration of the time interval and reinstantiation of the retry attempt.
9. A method according to claim 8 wherein defining the time interval further comprises defining the time interval such that the time interval progressively increases as the message is subjected to additional retry attempts.
10. A method according to claim 9 wherein the time interval progressively increases in a non-linear manner as the message is subjected to additional retry attempts.
11. A method according to claim 8 further comprising receiving publication of an event associated with the retry attempt following expiration of the time interval, wherein the retry attempt is reinstantiated in the primary memory in response to publication of the event.
12. A method according to claim 8 wherein reinstantiating the retry attempt is performed asynchronously.
13. A method according to claim 8 further comprising:
maintaining a count of retry attempts of a respective message; and
halting further retry attempts in response to the count of retry attempts satisfying a predefined limit.
14. A method according to claim 8 wherein the secondary persistent storage is implemented by a horizontally scaled set of machines.
15. A computer program product of a fault-tolerant retry subsystem, wherein the fault-tolerant retry subsystem comprises a processor and a primary memory accessed by the processor during execution of functions of the fault-tolerant retry subsystem, and wherein the computer program product comprises at least one non-transitory computer-readable storage medium having computer-executable program code portions stored therein, the computer-executable program code portions comprising program code instructions for:
receiving notification that delivery of the message to a subscriber has failed;
defining a time interval at which delivery of the message is to be retried, wherein the message is to be subjected to a plurality of retry attempts in an instance in which the delivery of the message to the subscriber continues to fail, and wherein defining the time interval comprises defining the time interval in a manner that is dependent upon which of the plurality of retry attempts is to be performed such that the time interval varies between at least some of the retry attempts;
causing a retry attempt to be scheduled in a secondary persistent storage in response to receipt of the notification that the delivery of the message has failed and following definition of the time interval, wherein the secondary persistent storage is slower to respond than the primary memory;
reinstantiating the retry attempt in the primary memory following scheduling of the retry attempt in the secondary persistent storage and in response to notification of the retry attempt by the secondary persistent storage following expiration of the time interval; and
causing the delivery of the message to be retried following expiration of the time interval and reinstantiation of the retry attempt.
16. A computer program product according to claim 15 wherein the program code instructions for defining the time interval further comprise program code instructions for defining the time interval such that the time interval progressively increases as the message is subjected to additional retry attempts.
17. A computer program product according to claim 16 wherein the time interval progressively increases in a non-linear manner as the message is subjected to additional retry attempts.
18. A computer program product according to claim 15 further comprising program code instructions for receiving publication of an event associated with the retry attempt following expiration of the time interval, wherein the retry attempt is reinstantiated in the primary memory in response to publication of the event.
19. A computer program product according to claim 15 wherein reinstantiation of the retry attempt is performed asynchronously.
20. A computer program product according to claim 15 wherein the secondary persistent storage is implemented by a horizontally scaled set of machines.