Intelligent predictive stream caching
View Patent ↗Some embodiments provide intelligent predictive stream caching for live, linear, or video-on-demand streaming content using prefetching, segmented caching, and request clustering. Prefetching involves retrieving streaming content segments from an origin server prior to the segments being requested by users. Prefetching live or linear streaming content segments involves continually reissuing requests to the origin until the segments are obtained or a preset retry duration is completed. Prefetching is initiated in response to a first request for a segment falling within a particular interval. Request clustering commences thereafter. Subsequent requests are queued until the segments are retrieved. Segmented caching involves caching segments for one particular interval. Segments falling within a next interval are not prefetched until a first request for one such segment in the next interval is received. Cached segments from the previous interval can be replaced in cache with segments for the current interval, thereby minimizing cache footprint utilization.
1. A method comprising:
receiving a first request for a first segment in a first time interval of live or linear streaming content;
retrieving said first segment from an origin server in response to the first request;
identifying in response to the first request for the first segment, at least a subsequent second segment of the live or linear streaming content within said first time interval and a different subsequent third segment of the live or linear streaming content within a different second time interval after said first time interval;
prefetching in response to said identifying, said subsequent second segment in the first time interval of the live or linear streaming content before receiving a user request for said second segment, wherein said prefetching comprises submitting at least two requests for the second segment to the origin server in response to a first of the at least two requests issuing to the origin server before the second segment publishes to the origin server;
serving the first segment in response to the first request upon receipt from the origin server;
storing said first and second segments in cache upon receipt from the origin server; and
responding to a plurality of different user requests for either of the first and second segments received after the first request without additional access to the origin server by passing in response to the plurality of user requests, the first and second segments either from said cache or upon receipt from the origin server.
2. The method of claim 1 further comprising receiving a first request for said third segment in said second time interval following the first time interval.
3. The method of claim 2 further comprising retrieving the third segment in the second time interval and prefetching a fourth segment after the third segment in the second time interval of the live or linear streaming content before receiving a user request for the fourth segment.
4. The method of claim 3 further comprising updating said cache by replacing cached copies of the first and second segments in the first time interval with the third and fourth segments in the second time interval.
5. The method of claim 1 further comprising receiving a set of user requests for the first segment after receiving said first request.
6. The method of claim 5 further comprising queuing the set of user requests without initiating any new requests to the origin server in response to any request of the set of user requests.
7. The method of claim 5 further comprising temporarily storing the set of user requests and responding to each request of the set of requests with said first segment upon retrieving the first segment from the origin server in response to the first request for the first segment.
8. The method of claim 1 further comprising receiving a request for the second segment after commencement of said prefetching.
9. The method of claim 8 further comprising holding the second segment request in a queue and responding to the second segment request upon receiving the second segment as a result of said prefetching initiated in response to the first request.
10. The method of claim 9 , wherein responding to the second segment request comprises periodically checking received responses from the origin server or cache for a copy of the second segment and serving the second segment when made available as a result of said prefetching.
11. A method comprising:
receiving over a digital network, a first request for a first segment in a first time interval of streaming content;
retrieving in response to the first request, the first segment and a first set of subsequent segments in the first time interval of the streaming content from an origin server;
storing the first segment and the first set of subsequent segments in the first time interval to a first of a plurality of partitions of cache, wherein each partition of the plurality of partitions stores different sets of objects;
receiving a second request for a different second segment in a second time interval of the streaming content, wherein the second time interval follows the first time interval;
retrieving in response to the second request, the second segment and a different second set of subsequent segments in the second time interval of the streaming content; and
overwriting the first partition of cache while retaining the different sets of objects in the other partitions of the plurality of partitions of cache in response to said retrieving of the second segment and the second set of subsequent segments, wherein said overwriting comprises replacing cached copies of the first segment and the first set of subsequent segments in the first partition of cache with the second segment and the second set of subsequent segments.
12. The method of claim 11 further comprising queuing a plurality of requests for the first segment in the first time interval that arrive after the first request, wherein said queuing comprises retaining the plurality of requests without issuing additional requests for the first segment to the origin server.
13. The method of claim 12 , wherein said queuing comprises responding to each request of the plurality of user requests with a copy of the first segment retrieved in response to the first request.
14. The method of claim 11 further comprising responding to each request of a plurality of requests for the first segment in the first time interval that arrives after the first request by distributing a cached copy of the first segment from the first partition of cache in response to each request of the plurality of requests.
15. The method of claim 11 wherein retrieving the first set of subsequent segments in the first time interval comprises prefetching the first set of subsequent segments in advance of receiving a user request for any segment of the first set of subsequent segments.
16. A method comprising:
configuring at least first and second distribution servers with distributing a particular live or linear content stream to a plurality of end users;
receiving at the first distribution server, a first request for a first segment in a first interval of the particular content stream;
submitting in response to the first request, requests for the first segment and at least one prefetched second segment in the first interval of the particular content stream from the first distribution server to an origin server, wherein the prefetched second segment is not requested by a user at time of said submitting and is prefetched in response to said first request for the first segment;
forwarding copies of the first segment and the prefetched second segment from the first distribution server to the second distribution server in response to the first distribution server receiving each of the first segment and the prefetched second segment from the origin server; and
populating a cache of the first distribution server and the second distribution server with the first segment and the prefetched second segment in response to said first request for the first segment received at the first distribution server.
17. The method of claim 16 further comprising passing in response to the first request a notification message from the first distribution server to the second distribution server, the notification message notifying the second distribution server that the first distribution server is retrieving segments in the first interval of the particular content stream.
18. The method of claim 17 further comprising receiving a second request for the first segment at the second distribution server after the second distribution server receives the notification message, and queuing the second request without the second distribution server requesting the first segment from the origin server.
19. The method of claim 18 further comprising responding to the second request with the second distribution server passing the first segment after receiving the first segment from the first distribution server as a result of said forwarding.
20. The method of claim 16 further comprising replacing said first segment from said cache with a third segment in a second interval of the particular content stream upon receiving a request for the third segment in the second interval.