HYBRID PLATFORM FOR CONTENT DELIVERY AND TRANSCODING
The subject matter herein generally relates to transcoding content, typically audio/video files though not limited to such, from one version to another in preparation for online streaming or other delivery to end users. Such transcoding may involve converting from one format to another (e.g., changing codecs or container formats), or creating multiple versions of an original source file in different bitrates, frame-sizes, or otherwise, to support distribution to a wide array of devices and to utilize performance-enhancing technologies like adaptive bitrate streaming. A transcoding platform is described herein that, in certain embodiments, leverages distributed computing techniques to transcode content in parallel across a platform of machines that are preferably idle or low-utilization resources of a content delivery network. The transcoding system also utilizes, in certain embodiments, improved techniques for segmenting the original source file so as to enable different segments to be sent to different machines for parallel transcodes.
1 . A system, comprising:
a plurality of proxy servers connected to a global computer network that operate to receive requests for content from clients and respond to the requests for content by sending the clients the content they requested;
a management server operable to receive a request to convert a file from a first version to a second version;
the management server operable to create at least first and second segments, each of the segments corresponding to a portion of the file, and send the first segment to a first one of the plurality of proxy servers and the second segment to a second one of the plurality of proxy servers, each of the first and second segments being sent with information about the requested conversion, so that the first and second segments are converted independently by the first and second proxy servers while the first and second proxy servers continue to respond to client requests for content;
wherein the plurality of proxy servers and the management server each comprise circuitry forming at least one processor and memory storing computer-readable instructions that when executed on the at least one processor will cause operation as specified above.
2 . The system of claim 1 , wherein the conversion involving changing at least one of:
(a) a codec used to encode data in the file,
(b) a container format of the file,
(c) one or more codec settings used to encode data in the file
(d) one or more container format settings for the file,
(e) a frame size for data in the file,
(f) an aspect ratio for data in the file,
(g) a bit-rate of encoded data in the file,
(h) an interlacing characteristic for data in the file,
(i) a frame rate for data in the file, and
(j) a picture resolution for data in the file.
3 . The system of claim 1 , wherein the conversion involves at least one of:
(a) changing one or more security characteristics of the file,
(b) applying a DRM scheme,
(c) applying encryption,
(d) applying a watermark, and
(e) applying a fingerprint.
4 . The system of claim 1 , wherein the first and second proxy servers were selected to participate in performing the requested conversion at least in part because their resource utilization related to servicing client requests for content was lower than that of other proxy servers.
5 . The system of claim 1 , wherein each of the plurality of proxy servers is operable to execute a first process providing a proxy function that services client requests for content, and a second process that performs conversions on files sent from the management server, the first process having priority over the second process.
6 . The system of claim 1 , wherein at least one of the plurality of proxy servers is operable to send the management server a message indicating that it will not perform the requested conversion, after that proxy server determines that its resource utilization related to servicing client requests exceeds a threshold.
7 . The system of claim 1 , wherein the management server is operable to identify proxy servers to use to perform the requested conversion by obtaining a list of one or more candidate proxy servers from a monitoring system associated with the plurality of proxy servers.
8 . The system of claim 1 , wherein the request to convert the file is associated with a priority, and the management server decides whether to use the plurality of proxy servers for performing the requested conversion based on the priority of the request.
9 . The system of claim 1 , wherein each of the first and second proxy servers operate to perform the requested conversion and return the results to the management server, which re-assembles the results into at least part of the second version of the file.
10 . The system of claim 1 , wherein the plurality of proxy servers are HTTP proxy servers and the content for which they receive client requests comprises any of HTML files, web page objects, and streaming media.
11 . The system of claim 1 , wherein file includes one or more of (i) audio data and (ii) video data.
12 . The system of claim 1 , further comprising a machine that makes the request to the management server to convert the file, the machine comprising any of: (a) a network storage system, (b) a server providing a user interface to content provider users of the system, and (c) one of the plurality of proxy servers.
13 . A method performed by one or more programmed computer machines that comprise circuitry forming one or more processors that execute computer program instructions, and that manage the conversion of content, the method comprising:
receiving a request to convert a file from a first version to a second version;
selecting first and second proxy servers from a plurality of proxy servers that are interconnected via a global computer network, and that are receiving requests for content from clients and responding to the requests for content by sending the clients the content they requested;
creating at least first and second segments, each of the segments corresponding to a portion of the file, and sending the first segment to the first proxy server and the second segment to the second proxy server, each of the first and second segments being sent with information about the requested conversion,
receiving a converted first segment from the first proxy server; and
receiving a converted second segment from the second proxy server
combining the converted first and second segments to form at least part of the second version of the file.
14 . The method of claim 13 , wherein the conversion involving changing at least one of:
(a) a codec used to encode data in the file,
(b) a container format of the file,
(c) one or more codec settings used to encode data in the file
(d) one or more container format settings for the file,
(e) a frame size for data in the file,
(f) an aspect ratio for data in the file,
(g) a bit-rate of encoded data in the file,
(h) an interlacing characteristic for data in the file,
(i) a frame rate for data in the file, and
(j) a picture resolution for data in the file.
15 . The method of claim 13 , wherein the conversion involves at least one of:
(a) changing one or more security characteristics of the file,
(b) applying a DRM scheme,
(c) applying encryption,
(d) applying a watermark, and
(e) applying a fingerprint.
16 . The method of claim 13 , wherein the first and second proxy servers are selected at least in part because their resource utilization related to servicing client requests for content is lower than that of other proxy servers.
17 . The method of claim 13 , further comprising: receiving from one of the plurality of proxy servers a message indicating that it will not convert a particular segment because its resource utilization related to servicing client requests exceeds a threshold.
18 . The method of claim 13 , further comprising: identifying proxy servers to use to perform the requested conversion by obtaining a list of one or more candidate proxy servers from a monitoring system associated with the plurality of proxy servers.
19 . The method of claim 13 , wherein the request to convert the file is associated with a priority, and further comprising deciding whether to use the plurality of proxy servers for performing the requested conversion based on the priority of the request.
20 . The method of claim 13 , wherein the plurality of proxy servers are HTTP proxy servers and the content for which they receive client requests comprises any of HTML files, web page objects, and streaming media.
21 . The method of claim 13 , wherein the file includes one or more of (i) audio data and (ii) video data.
22 . The method of claim 13 , further comprising receiving the request to convert the file from any of: (a) a network storage system, (b) a server providing a user interface to content provider users of the system and (c) one of the plurality of proxy servers.
23 . A method performed by programmed computer machines that comprise circuitry forming one or more processors that execute computer program instructions, comprising:
with a plurality of proxy servers that are connected to a global computer network, receiving for content from clients and responding to the requests for content by sending the clients the content they requested;
at a first proxy server selected from the plurality of proxy servers, receiving a request to convert a first segment of a file from a first version to a second version, and instructions about the conversion to be performed;
at a second proxy server selected from the plurality of proxy servers, receiving a request to convert a second segment of the file from a first version to a second version and instructions about the conversion to be performed;
the first proxy server converting the first segment from the first version to the second version while continuing to response to client requests for content, as long as the load on the first proxy server due to the client requests for content does not exceed a threshold;
the first proxy server sending the second version of the first segment to at least one server managing the conversion;
the second proxy server converting the second segment from the first version to the second version while continuing to response to client requests for content, as long as the load on the second proxy server due to the client requests for content does not exceed a threshold;
the second proxy server sending the second version of the first segment to the at least one server managing the conversion.
24 . The method of claim 23 , wherein the conversion involving changing at least one of:
(k) a codec used to encode data,
(l) a container format,
(m) one or more codec settings,
(n) one or more container format settings,
(o) a frame size,
(p) an aspect ratio,
(q) a bit-rate of encoded data,
(r) an interlacing characteristic,
(s) a frame rate, and
(t) a picture resolution.
25 . The method of claim 23 , wherein the conversion involves at least one of:
(f) changing one or more security characteristics,
(g) applying a DRM scheme,
(h) applying encryption,
(i) applying a watermark, and
(j) applying a fingerprint.