Processing resource management system and methods
The present invention discloses a system and methods for parallel processing of multiple processing job requests; the system may include a server for receiving a job request, an algorithm for segmenting the job request to a few sub jobs and a few processors for processing the few sub jobs in parallel. Each sub job contains a few frames to be processed by the job processors and the outputs of few job processors are combined into a single output. The invention further discloses methods for proportional allocation of job segments and an optimization algorithm to automatically assign job requests and to adapt the resources of the system to meet customers demand according to predefined criteria.
1. A method for parallel rendering of multiple video processing jobs, the method comprising:
receiving multiple video processing jobs, each video processing job comprising multiple video frames;
segmenting each video processing job into sub jobs, each sub job comprising a pack of video frames selected from the multiple video frames;
rendering the sub jobs by multiple sub job processors, thereby generating multiple segmented video products in parallel;
encoding the multiple segmented video products by the multiple sub job processors; and
aggregating the multiple segmented video products into a final video product, wherein said segmenting utilizes an optimization algorithm configured for:
pre-processing the multiple video processing jobs for evaluating processing loads of a few frames in the multiple video frames; and
packing frames from the multiple video frames into the packs of video frames based on the evaluated processing loads of the few frames, such that a processing load of at least a few sub jobs is proportional to a processing power of the sub job processors by which said a few subs job are rendered.
2. The method according to claim 1 , wherein a priority value is assigned to each of the multiple video processing jobs.
3. The method according to claim 1 , wherein a priority value is composed by taking into consideration a client priority value requested by a client and system obligations to meet the client priority value.
4. The method according to claim 1 , wherein the optimization algorithm is a learning algorithm, configured to learn from past experience, to consider real-time parameters and consider a processing power of each of the sub job processers.
5. The method according to claim 1 , wherein parent job processors, functionally associated with at least one parent job dispatcher and each comprising multiple sub job processors, are configured to receive video processing jobs from the parent job dispatcher and to segment the video processing jobs into sub jobs to be processed by the multiple sub job processors, wherein the parent job processors are located at different geographic locations.
6. The method according to claim 5 , wherein at least a few sub-job processors are private-user processors.
7. The method according to claim 1 , wherein the optimization algorithm takes into consideration cost effective parameters.
8. The method according to claim 1 , wherein a rendezvous server is configured to add or remove one or more sub job processors to or from the multiple sub job processors dynamically.
9. The method according to claim 1 , further comprising interfacing with a user using an intuitive user interface provided via a web-site.
10. A system for parallel rendering of multiple video processing jobs, the system comprising:
a server configured for
receiving multiple video processing jobs comprising multiple video frames;
utilizing an optimization algorithm, segmenting each processing job into multiple sub jobs such that each sub job comprises a pack of frames selected from the multiple video frames, to be rendered into multiple segmented video products using multiple sub job processors, and
aggregating the multiple segmented video products into a final video product; and
multiple sub job processors configured for rendering the multiple segmented sub jobs and for encoding each sub job to a segmented video product by the multiple sub job processors in parallel, wherein said optimization algorithm comprises:
pre-processing the multiple video processing jobs for evaluating processing loads of a few frames in the multiple video frames; and packing frames from the multiple video frames into the packs of video frames based on the evaluated processing loads of the few frames, such that a processing load of at least a few sub jobs is proportional to a processing power of the sub job processors by which said at least a few sub jobs are rendered.
11. The system according to claim 10 , wherein a rendezvous server is configured to add or remove one or more sub job processors to the multiple sub job processors dynamically.
12. The system according to claim 10 , wherein the multiple sub job processors are located at different geographic locations and are associated with a job dispatcher.