Multi-tiered storage systems and methods for adaptive content streaming
The disclosed technology includes techniques for efficiently streaming media content from a multi-tiered storage system. An example implementation may be used for adaptive HTTP streaming of video segments and other content. In some implementations, flash memory SSDs (SLC or MLC) may form an intermediate cache layer between a first layer of DRAM cache and third layer of HDDs. Significant architectural elements of various implementations include optimal write granularity to overcome the write amplification effect of flash memory SSDs and a QoS-sensitive caching strategy that monitors the activity of the flash memory SSDs to ensure that video streaming performance is not hampered by the caching activity.
1. A multi-tiered storage system for streaming media content segments comprising:
a plurality of storage levels including:
a RAM buffer pool embodied in a first level of RAM cache, the RAM buffer pool comprising an array of memory blocks of various sizes, each block storing data for a media segment file;
an SSD buffer pool embodied in a second level of SSD cache, wherein an address space associated with the SSD cache is defined by equal-size blocks of an optimal block size associated with the SSD cache, and wherein write operations to the SSD cache are requested with the optimal block size and aligned on a block boundary associated with the SSD cache; and
a third level of HDD storage;
a processor operatively coupled to the plurality of storage layers; and
a RAM monitor thread adapted to monitor a tail part of the RAM buffer pool, the tail part referred to as a monitor room;
wherein a size associated with the monitor room is about 10% of a total number of segments in the RAM buffer pool;
wherein media segment requests that miss the RAM cache and SSD cache are served by the HDD storage;
wherein media segments read from the HDD storage are output to the RAM buffer pool and returned to the processor from the RAM buffer pool; and
wherein media segments read from the SSD are not inserted into the RAM buffer pool but are returned to the processor.
2. The system of claim 1 , wherein the SSD cache is at least partially implemented in MLC flash memory.
3. The system of claim 1 , wherein no same data segment is stored in both the RAM cache and the SSD cache.
4. The system of claim 1 , wherein segments cached in the RAM buffer pool are ordered according to a least recently used block replacement policy.
5. The system of claim 1 further comprising:
a ring buffer adapted for storing media segments that are candidates for being written into the SSD cache; and
an SSD feed thread adapted to copy segments from the ring buffer to the SSD cache in units of the optimal block size.
6. The system of claim 5 , wherein the SSD feed thread is delayed from making write requests to the SSD cache responsive to determining the SSD cache is busy serving read requests.
7. The system of claim 6 , wherein the determining the SSD cache is busy serving read requests is based on a predetermined usage threshold.
8. The system of claim 5 , wherein the SSD feed thread is further adapted to discard segments from the ring buffer responsive to determining the SSD cache is full.
9. The system of claim 5 , wherein blocks are evicted from the SSD cache based on an evict-ahead policy.
10. The system of claim 1 , wherein the optimal block size associated with the SSD cache is empirically determined by:
outputting a first plurality of random write commands associated with a first request size to the SSD cache;
outputting a second plurality of sequential write commands associated with the first request size to SSD cache; and
comparing latency distributions associated with the first plurality of random write commands and the second plurality of sequential write commands.
11. The system of claim 1 , wherein the SSD cache is at least partially implemented in SLC flash memory.
12. A multi-tiered storage system for streaming media content segments comprising:
a plurality of storage levels including:
a RAM buffer pool embodied in a first level of RAM cache, the RAM buffer pool comprising an array of memory blocks of various sizes, each block storing data for a media segment file;
an SSD buffer pool embodied in a second level of SSD cache, wherein an address space associated with the SSD cache is defined by equal-size blocks of an optimal block size associated with the SSD cache, and wherein write operations to the SSD cache are requested with the optimal block size and aligned on a block boundary associated with the SSD cache; and
a third level of HDD storage; and
a processor operatively coupled to the plurality of storage layers;
wherein media segment requests that miss the RAM cache and SSD cache are served by the HDD storage;
wherein media segments read from the HDD storage are output to the RAM buffer pool and returned to the processor from the RAM buffer pool;
wherein media segments read from the SSD are not inserted into the RAM buffer pool but are returned to the processor; and
wherein the optimal block size associated with the SSD cache is empirically determined by:
outputting a first plurality of random write commands associated with a first request size to the SSD cache;
outputting a second plurality of sequential write commands associated with the first request size to SSD cache; and
comparing latency distributions associated with the first plurality of random write commands and the second plurality of sequential write commands based on Kullback-Leibler divergence.
13. A method of reducing write amplification with SSDs, comprising:
determining, experimentally by a processor, an optimal block size associated with an SSD, the determining comprising:
outputting a first plurality of random write commands associated with a first request size to the SSD;
outputting a second plurality of sequential write commands associated with the first request size to the SSD;
comparing, based on Kullback-Leibler divergence, latency distributions associated with the first plurality of random write commands and the second plurality of sequential write commands;
defining a logical address space associating the SSD with allocation blocks of a same size as the optimal block size; and
performing, by the processor, one or more writes to flash memory, each write at the optimal block size and aligned on a block boundary associated with the SSD.
14. The method of claim 13 , wherein the optimal block size is selected responsive to determining a Kullback-Leibler divergence value equal to or less than 0.1.
15. The method of claim 13 , wherein the optimal block size is selected responsive to determining a Kullback-Leibler divergence value equal to or less than a predetermined threshold.
16. A non-transitory computer-readable medium that stores instructions that, when executed by a processor of a computing device, causes the computing device to perform a method of reducing write amplification with SSDs, comprising:
determining, experimentally by the processor, an optimal block size associated with an SSD, the determining comprising:
outputting a first plurality of random write commands associated with a first request size to the SSD;
outputting a second plurality of sequential write commands associated with the first request size to the SSD;
comparing, based on Kullback-Leibler divergence, latency distributions associated with the first plurality of random write commands and the second plurality of sequential write commands;
defining a logical address space associating the SSD with allocation blocks of a same size as the optimal block size;
performing, by the processor, one or more writes to flash memory, each write at the optimal block size and aligned on a block boundary associated with the SSD.