Just-in-time distributed video cache
A video optimizer receives a request to optimize a source file and stream the optimized file. The request may be accompanied by a key uniquely identifying the original source file to be optimized and an optimized version of the source file to stream to the client. The key uniquely identifies the source video file as it exists on an origin server to prevent transmission of modified source video and the optimized version of the source file by optimization parameters to ensure compatibility with the user device. The video optimizer queries a cache database in response to the received request, which contains references keys and locations of optimized source files among local caches in a cluster of video optimizers. The cache database responds indicates whether a matching optimized version of the source file exists in a local cache of a given video optimizer in the cluster.
1. A method for optimized video caching comprising:
maintaining, by a video optimizer, a cache database including a plurality of reference keys corresponding to a plurality of optimized versions of one or more source files stored on a plurality of video optimization servers, each reference key specifying parameters of an optimized version of a source file and including an identifier of the source file and an identifier of a video optimization server storing the optimized version of the source file;
receiving a request at the video optimizer to optimize an original source file from a client device, the original source file stored on an origin server and the request including a key specifying parameters for optimizing the original source file and an identifier of the original source file;
generating a query, based on the key, to the cache database;
receiving, from the cache database, a response indicating whether the key matches a reference key maintained in the database; and
transmitting to the client device, responsive to the cache database response indicating the key matches the reference key, a redirect request comprising the key and an address of the video optimization server specified by the reference key for the client device to request the optimized version of the original source file.
2. The method of claim 1 , further comprising, responsive to the cache database response indicating that the key matches the reference key :
transmitting, to the video optimization server specified by the reference key, a request for the optimized version of the original source file stored on the video optimization server.
3. The method of claim 2 , further comprising:
storing the portion of the received optimized version of the source file.
4. The method of claim 2 , further comprising:
transmitting, with the request for the optimized version of the original source file stored on the video optimization server specified by the reference key, a request to duplicate the optimized source file stored on the video optimization server on the video optimizer; and
transmitting, to the cache database, instructions for updating the cache database in response to storing the optimized version of the original source file stored in the video optimizer.
5. The method of claim 4 , further comprising:
querying the origin server to determine whether the stored optimized version of the original source file matches the video length of the original source file.
6. The method of claim 1 , wherein the key comprises a source signature, the source signature being a hash of a portion of the original source file.
7. The method of claim 1 , wherein the cache database comprises data entries identifying a plurality of optimization servers storing an optimized version of the original source file, the entries being modified through instructions transmitted to the cache database to update a location where the optimized version of the original source file is stored.
8. The method of claim 1 , wherein the stored optimized version of the original source file comprises a predefined segment of the entire generated optimized source file.
9. The method of claim 1 , further comprising, responsive to the cache database response indicating that the key does not match any reference key for an optimized version of the original source file:
transmitting, to the origin server, a request for the original source file;
streaming, from the origin server, the original source file;
generating the optimized version of the original source file by transcoding the stream based on predetermined transcoding settings and storing the optimized version of the original source file;
streaming the optimized version of the original source file for playback on the client device; and
transmitting, to the cache database, instructions for updating the cache database with a reference key corresponding to the optimized source file stored at the optimization server.
10. The method of claim 1 , wherein the key matches the reference key responsive to the identifier of the original source file in the key matching the identifier of the source file in the reference key and the optimization parameters specified by the key matching the parameters of an optimized version of the source file.
11. A system for optimized video caching, comprising:
a processor; and
a non-transitory computer-readable storage medium storing executable computer program instructions that when executed causes the processor to:
maintain, by a video optimizer, a cache database including a plurality of reference keys corresponding to a plurality of optimized versions of one or more source files stored on a plurality of video optimization servers, each reference key specifying parameters of an optimized version of a source file and including an identifier of the source file and an identifier of a video optimization server storing the optimized version of the source file;
receive a request to optimize an original source file from a client device, the original source file stored on an origin server and the request including a key specifying parameters for optimizing the original source file and an identifier of the original source file;
generate a query, based on the key, to the cache database
receive, from the cache database, a response indicating whether the key matches a reference key maintained in the database; and
transmit to the client device, responsive to the cache database response indicating the key matches the reference key, a redirect request comprising the key and an address of the video optimization server specified by the reference key for the client device to request the optimized version of the original source file.
12. The system of claim 11 , further comprising instructions that cause the processor to, responsive to the cache database response indicating that the key matches the reference key:
transmit, to the video optimization server specified by the reference key, a request for the optimized version of the original source file stored on the video optimization server.
13. The system of claim 11 , further comprising instructions that cause the processor to, responsive to the cache database response indicating that the key does not match any reference key for an optimized version of the original source file:
transmit, to the origin server, a request for the original source file;
stream, from the origin server, the original source file;
generate the optimized version of the original source file by transcoding the stream based on predetermined transcoding settings and storing the optimized version of the original source file;
stream the optimized version of the original source file for playback on the client device; and
transmit, to the cache database, instructions for updating the cache database with a reference key corresponding to the optimized source file stored at the optimization server.
14. The system of claim 11 , wherein the key matches the reference key responsive to the identifier of the original source file in the key matching the identifier of the source file in the reference key and the optimization parameters specified by the key matching the parameters of an optimized version of the source file.