Staggered payload relayer for pipelining digital payloads across network services
This disclosure describes a payload pipeline system that, as part of an inter-network facilitation system, can facilitate digital payload throughput between network components using a staggered payload relayer. For example, the disclosed systems utilize custom tooling to implement a staggered payload relayer as an intelligent conduit or pipeline that operates as a publisher-subscriber paradigm system (as opposed to a data stream system). In some cases, the disclosed systems can utilize synchronous or asynchronous communication to receive digital payloads for events from publisher network components and to make the digital payloads available within a payload queue maintained by the staggered payload relayer (for access by subscriber network components).
1 . A system comprising:
at least one processor; and
a non-transitory computer readable medium comprising instructions that, when executed by the at least one processor, cause the system to:
receive, from a publisher network component of a network-based system comprising a plurality of compartmentalized network components comprising executable server software dedicated to performing respective functions of the network-based system, a digital payload for a staggered payload relayer hosted on a server to relay, the publisher network component comprising a network component of the plurality of compartmentalized network components that generates or provides the digital payload;
add the digital payload to a payload queue maintained by the staggered payload relayer and comprising a plurality of digital payloads received from various publisher network components of the network-based system;
identify, within the network-based system, a subscriber network component subscribed to receive the digital payload from the payload queue; and
provide, in response to identifying the subscriber network component and utilizing the staggered payload relayer, the digital payload to the subscriber network component subscribed to receive the digital payload via staggered payload distribution.
2 . The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:
facilitate publication of digital payloads within the payload queue utilizing the staggered payload relayer as an intermittent message queue system for staggered payload queuing; and
facilitate subscription to the digital payloads within the payload queue utilizing the staggered payload relayer as an intermittent message notification system for staggered payload distribution.
3 . The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:
generate a concurrent payload group by combining multiple digital payloads from the payload queue together; and
provide the concurrent payload group to an intermittent message notification system within a single transmission to distribute the multiple digital payloads to subscriber network components.
4 . The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:
convert staggered digital payloads within the payload queue to a payload stream utilizing a stagger-to-stream converter; and
provide the payload stream to a payload stream broadcaster for broadcasting the payload stream to network components.
5 . The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:
utilize the staggered payload relayer to perform synchronous payload transmission by, in response to providing the digital payload to the subscriber network component, waiting for a confirmation from the subscriber network component before providing a subsequent digital payload; or
utilize the staggered payload relayer to perform asynchronous payload transmission by providing the digital payload to the subscriber network component without waiting for a confirmation from the subscriber network component.
6 . The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:
generate a debugging wrapper utilizing the staggered payload relayer to include as part of the digital payload; and
provide the debugging wrapper with the digital payload for tracing the digital payload among network components.
7 . The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to maintain the digital payload within the payload queue of the staggered payload relayer for a set duration.
8 . A method comprising:
receiving, from a publisher network component of a network-based system comprising a plurality of compartmentalized network components comprising executable server software dedicated to performing respective functions of the network-based system, a digital payload for a staggered payload relayer hosted on a server to relay, the publisher network component comprising a network component of the plurality of compartmentalized network components that generates or provides the digital payload;
adding the digital payload to a payload queue maintained by the staggered payload relayer and comprising a plurality of digital payloads received from various publisher network components of the network-based system;
identifying, within the network-based system, a subscriber network component subscribed to receive the digital payload from the payload queue; and
providing, in response to identifying the subscriber network component and utilizing the staggered payload relayer, the digital payload to the subscriber network component subscribed to receive the digital payload via staggered payload distribution.
9 . The method of claim 8 , further comprising:
facilitating publication of digital payloads within the payload queue utilizing the staggered payload relayer as an intermittent message queue system for staggered payload queuing; and
facilitating subscription to the digital payloads within the payload queue utilizing the staggered payload relayer as an intermittent message notification system for staggered payload distribution.
10 . The method of claim 8 , further comprising:
generating a concurrent payload group by combining multiple digital payloads from the payload queue together; and
providing the concurrent payload group to an intermittent message notification system within a single transmission to distribute the multiple digital payloads to subscriber network components.
11 . The method of claim 8 , further comprising:
converting staggered digital payloads within the payload queue to a payload stream utilizing a stagger-to-stream converter; and
providing the payload stream to a payload stream broadcaster for broadcasting the payload stream to network components.
12 . The method of claim 8 , further comprising:
utilizing the staggered payload relayer to perform synchronous payload transmission by, in response to providing the digital payload to the subscriber network component, waiting for a confirmation from the subscriber network component before providing a subsequent digital payload; or
utilizing the staggered payload relayer to perform asynchronous payload transmission by providing the digital payload to the subscriber network component without waiting for a confirmation from the subscriber network component.
13 . The method of claim 8 , further comprising:
generating a debugging wrapper utilizing the staggered payload relayer to include as part of the digital payload; and
providing the debugging wrapper with the digital payload for tracing the digital payload among network components.
14 . The method of claim 8 , further comprising maintaining the digital payload within the payload queue of the staggered payload relayer for a set duration.
15 . A non-transitory computer readable medium comprising instructions that, when executed by at least one processor, cause a computing device to:
receive, from a publisher network component of a network-based system comprising a plurality of compartmentalized network components comprising executable server software dedicated to performing respective functions of the network-based system, a digital payload for a staggered payload relayer hosted on a server to relay, the publisher network component comprising a network component of the plurality of compartmentalized network components that generates or provides the digital payload;
add the digital payload to a payload queue maintained by the staggered payload relayer and comprising a plurality of digital payloads received from various publisher network components of the network-based system;
identify, within the network-based system, a subscriber network component subscribed to receive the digital payload from the payload queue; and
provide, in response to identifying the subscriber network component and utilizing the staggered payload relayer, the digital payload to the subscriber network component subscribed to receive the digital payload via staggered payload distribution.
16 . The non-transitory computer readable medium of claim 15 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
facilitate publication of digital payloads within the payload queue utilizing the staggered payload relayer as an intermittent message queue system for staggered payload queuing; and
facilitate subscription to the digital payloads within the payload queue utilizing the staggered payload relayer as an intermittent message notification system for staggered payload distribution.
17 . The non-transitory computer readable medium of claim 15 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
generate a concurrent payload group by combining multiple digital payloads from the payload queue together; and
provide the concurrent payload group to an intermittent message notification system within a single transmission to distribute the multiple digital payloads to subscriber network components.
18 . The non-transitory computer readable medium of claim 15 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
convert staggered digital payloads within the payload queue to a payload stream utilizing a stagger-to-stream converter; and
provide the payload stream to a payload stream broadcaster for broadcasting the payload stream to network components.
19 . The non-transitory computer readable medium of claim 15 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
utilize the staggered payload relayer to perform synchronous payload transmission by, in response to providing the digital payload to the subscriber network component, waiting for a confirmation from the subscriber network component before providing a subsequent digital payload; or
utilize the staggered payload relayer to perform asynchronous payload transmission by providing the digital payload to the subscriber network component without waiting for a confirmation from the subscriber network component.
20 . The non-transitory computer readable medium of claim 15 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
generate a debugging wrapper utilizing the staggered payload relayer to include as part of the digital payload; and
provide the debugging wrapper with the digital payload for tracing the digital payload among network components.