Optimal strategy for data replay in a complex multiplexing network
The technology disclosed relates to accelerating replay access to numerous streams via a channel aggregation hierarchy, muxing events from input buffers that receive event streams, into a physically aggregated stream in memory. A subscriber accesses events from the streams via the channel hierarchy; demuxing the physical aggregated stream and caching the events from the physical aggregated stream into event cache memory; caching references to selected events, which belong to active channels that have subscribers, in a caching structure including channel cache lists of the references, the caching structure including stream level channel cache lists, intermediate level channel cache lists, and, if the physical aggregate channel has subscribers, a physical aggregate channel cache list; and receiving a replay access request that specifies a channel and that specifies a starting point for the replay, and responsively delivering events from the event cache memory using a cache list corresponding to the specified channel.
1. A method of accelerating replay access to numerous streams via a channel aggregation hierarchy, including:
muxing events from at least 100 input buffers that receive event streams, into a physical aggregated stream in memory, wherein a subscriber is permitted to access events from the streams via a stream level channel, a physical aggregate level channel or an intermediate level channel that aggregates stream level channels and contributes to the physical aggregate level channel;
demuxing the physical aggregated stream and caching the events from the physical aggregated stream into event cache memory;
caching references to selected events, which belong to active channels that have subscribers, in a caching structure including channel cache lists of the references, the caching structure representing the channel aggregation hierarchy, including stream level channel cache lists, intermediate level channel cache lists, and, if the physical aggregate channel has subscribers, a physical aggregate channel cache list; and
receiving a replay access request that specifies a channel at one of the stream level, intermediate level or physical aggregate level and that specifies a starting point for the replay, and responsively delivering events from the event cache memory using a cache list corresponding to the specified channel included in the caching structure.
2. The method of claim 1 , further including repeating the demuxing at ten to two hundred millisecond intervals.
3. The method of claim 1 , wherein eighty five to ninety five percent of events delivered responsive to replay access requests are already present in the cache, further including: responding to the replay access requests and responsively delivering events from the event cache memory using the starting point established for the replay and the events present in the event cache memory, without requiring real-time redundant demuxing of requested events.
4. The method of claim 1 , wherein the cache lists include first cache list for a low-throughput stream with a streaming rate of zero events to one event per minute and a second cache list for a high-throughput stream with a streaming rate of one to 10 million events per minute.
5. The method of claim 1 , further including:
batching cache miss-causing replay access requests that request events from a same physical aggregate channel, wherein a cache miss-causing replay access request is a request for events not already present in the cache; and
demuxing events in the physical aggregate channel, producing demuxed events, and caching the demuxed events into event cache memory responsive to the batch of cache miss-causing replay access requests.
6. The method of claim 1 , further including:
tracking a count of active subscribers for each channel; and
using the count to identify a number of channels for which to maintain the cache lists.
7. The method of claim 1 , further including:
when a count of active subscribers for a channel diminishes to zero, continue maintaining a corresponding cache list for a predetermined time period to allow for subscribers to re-subscribe.
8. The method of claim 1 , wherein the cache lists are stored in a Java Virtual Machine heap.
9. The method of claim 1 , further including demuxing events in the physical aggregate channel, producing demuxed events, and caching the demuxed events into event cache memory on a batch-by-batch basis.
10. A system, including memory loaded with computer program instructions and one or more processors operable to execute instructions stored in the memory, the instructions, when executed on the processors, implement a method of accelerating replay access to numerous streams via a channel aggregation hierarchy, including:
muxing events from at least 50 input buffers that receive event streams, into a physical aggregated stream in memory, wherein a subscriber is permitted to access events from the streams via a stream level channel, a physical aggregate level channel or an intermediate level channel that aggregates stream level channels and contributes to the physical aggregate level channel;
demuxing the physical aggregated stream and caching the events from the physical aggregated stream into event cache memory;
caching references to selected events, which belong to active channels that have subscribers, in a caching structure including channel cache lists of the references, the caching structure representing the channel aggregation hierarchy, including stream level channel cache lists, intermediate level channel cache lists, and, if the physical aggregate channel has subscribers, a physical aggregate channel cache list; and
receiving a replay access request that specifies a channel at one of the stream level, intermediate level or physical aggregate level and that specifies a starting point for the replay, and responsively delivering events from the event cache memory using a cache list corresponding to the specified channel included in the caching structure.
11. The system of claim 10 , further including:
batching cache miss-causing replay access requests that request events from a same physical aggregate channel, wherein a cache miss-causing replay access request is a request for events not already present in the cache; and
demuxing events in the physical aggregate channel, producing demuxed events, and caching the demuxed events into event cache memory responsive to the batch of cache miss-causing replay access requests.
12. The method of claim 10 , further including demuxing events in the physical aggregate channel, producing demuxed events, and caching the demuxed events into event cache memory on a batch-by-batch basis.
13. A tangible non-transitory computer readable storage medium impressed with computer program instructions executable by a processor, the instructions, when executed on a processor, implement a method of accelerating replay access to numerous streams via a channel aggregation hierarchy, comprising:
muxing events from at least 100 input buffers that receive event streams, into a physical aggregated stream in memory, wherein a subscriber is permitted to access events from the streams via a stream level channel, a physical aggregate level channel or an intermediate level channel that aggregates stream level channels and contributes to the physical aggregate level channel;
demuxing the physical aggregated stream and caching the events from the physical aggregated stream into event cache memory;
caching references to selected events, which belong to active channels that have subscribers, in a caching structure including channel cache lists of the references, the caching structure representing the channel aggregation hierarchy, including stream level channel cache lists, intermediate level channel cache lists, and, if the physical aggregate channel has subscribers, a physical aggregate channel cache list; and
receiving a replay access request that specifies a channel at one of the stream level, intermediate level or physical aggregate level and that specifies a starting point for the replay, and responsively delivering events from the event cache memory using a cache list corresponding to the specified channel included in the caching structure.