Just-in-time distributed video cache
A video optimizer receives a request to optimize a source file and stream the optimized file. The request may be accompanied by a key uniquely identifying the original source file to be optimized and an optimized version of the source file to stream to the client. The key uniquely identifies the source video file as it exists on an origin server to prevent transmission of modified source video and the optimized version of the source file by optimization parameters to ensure compatibility with the user device. The video optimizer queries a cache database in response to the received request, which contains references keys and locations of optimized source files among local caches in a cluster of video optimizers. The cache database responds indicates whether a matching optimized version of the source file exists in a local cache of a given video optimizer in the cluster.
1. A computer-implemented method for streaming video, the method comprising:
receiving a request to optimize an original source file from a client computing device, the request including a key specifying parameters to optimize the original source file;
identifying, based on the key, a first portion of an optimized version of the original source file in a local cache of a video optimizer;
generating a query, based on the key, to a database maintaining a plurality of reference keys corresponding to a plurality of optimized versions of source files stored on a plurality of other video optimizers, each reference key specifying parameters of an optimized version of a source file;
receiving, from the database, a response indicating whether the key matches a reference key;
receiving from the database, in response to a reference key matching the key, the address of another video optimizer storing a segment of the requested optimized version of the original source file;
stitching the first portion of the optimized version of the original source file with the segment of the optimized version of the original source file stored on the other video optimizer; and
streaming the stitched optimized version of the original source file for playback on the client computing device.
2. The method of claim 1 , further comprising:
storing a segment of the stitched optimized version of the original source file in a cache of the video optimizer; and
transmitting, to the database, instructions to update the database in response to storing the segment of the stitched optimized version of the original source file stored in the video optimizer.
3. The method of claim 1 , wherein the received response indicates that different segments of the optimized version of the same original source file are stored on a plurality of different video optimizers.
4. The method of claim 3 , further comprising:
transmitting, to the plurality of different video optimizers, a request for the segment of the optimized version of the original source file stored on each video optimizer; and
stitching the different segments of the optimized version of the same original source file with the first portion of the original source file.
5. The method of claim 1 , wherein the key comprises a source content signature, the source content signature being a hash of the original source file.
6. The method of claim 1 , wherein the database comprises data entries identifying a plurality of video optimizers storing an optimized version of the original source file, the entries being modified through instructions transmitted to the database to update a location where the optimized version of the original source file is stored.
7. The method of claim 1 , further comprising:
determining a second segment of the optimized version of the original source file is not stored by the plurality of other video optimizers;
transcoding, responsive to the determination, the second segment of the original source file;
storing the transcoded second segment in the local cache of the video optimizer; and
stitching the transcoded second segment with the stitched optimized version of the original source file.
8. The method of claim 7 , further comprising transmitting to the database, instructions to update the database with a reference key corresponding to the transcoded second segment of the optimized version of the source file stored at the video optimizer.
9. The method of claim 7 , wherein transcoding the second segment of the original source file comprises:
identifying a third segment of the optimized version of the original source file, subsequent to the second segment, that is stored by one of the plurality of other video optimizers; and
transcoding the original source file from a start of the second segment to a start of the third segment.
10. The method of claim 1 , wherein the first portion is maintained in the local cache of the video optimizer after one or more purges of the local cache.
11. A system for streaming optimized video to a client device, comprising:
a processor; and
a non-transitory computer-readable storage medium storing executable computer program instructions that, when executed by the processor, causes the processor to:
receive a request to optimize an original source file from a client computing device, the request including a key specifying parameters to optimize the original source file;
identify, based on the key, a first portion of an optimized version of the original source file in a local cache of a video optimizer;
generate a query, based on the key, to a database maintaining a plurality of reference keys corresponding to a plurality of optimized versions of source files stored on a plurality of other video optimizers, each reference key specifying parameters of an optimized version of a source file;
receive, from the database, a response indicating whether the key matches a reference key;
receive from the database, in response to a reference key matching the key, the address of another video optimizer storing a segment of the requested optimized version of the original source file;
stitch the first portion of the optimized version of the original source file with the segment of the optimized version of the original source file stored on the other video optimizer; and
stream the stitched optimized version of the original source file for playback on the client computing device.
12. The system of claim 11 , further comprising instructions that cause the processor to:
store a segment of the stitched optimized version of the original source file in a cache of the video optimizer; and
transmit, to the database, instructions to update the database in response to storing the segment of the stitched optimized version of the original source file stored in the video optimizer.
13. The system of claim 11 , wherein the received response indicates that different segments of the optimized version of the same original source file are stored on a plurality of different video optimizers.
14. The system of claim 13 , further comprising instructions that cause the processor to:
transmit, to the plurality of different video optimizers, a request for the segment of the optimized version of the original source file stored on each video optimizer; and
stitch the different segments of the optimized version of the same original source file with the first portion of the original source file.
15. The system of claim 11 , wherein the key comprises a source content signature, the source content signature being a hash of the original source file.
16. The system of claim 11 , wherein the database comprises data entries identifying a plurality of video optimizers storing an optimized version of the original source file, the entries being modified through instructions transmitted to the database to update a location where the optimized version of the original source file is stored.
17. The system of claim 11 , further comprising instructions that cause the processor to:
determine a second segment of the optimized version of the original source file is not stored by the plurality of other video optimizers;
transcode, responsive to the determination, the second segment of the original source file;
store the transcoded second segment in the local cache of the video optimizer; and
stitch the transcoded second segment with the stitched optimized version of the original source file.
18. The system of claim 17 , further comprising instructions that cause the processor to transmit to the database, instructions for updating the database with a reference key corresponding to the transcoded second segment of the optimized version of the source file stored at the video optimizer.
19. The system of claim 17 , wherein transcoding the second segment of the original source file comprises:
identifying a third segment of the optimized version of the original source file, subsequent to the second segment, that is stored by one of the plurality of other video optimizers; and
transcoding the original source file from a start of the second segment to a start of the third segment.
20. The system of claim 11 , wherein the first portion is maintained in the local cache of the video optimizer after one or more purges of the local cache.