IP Library Granted Patent US 11,087,429
Granted Patent B2
US 11,087,429 · App. 16/517,155 · Granted Aug 10, 2021

Run-time optimization of media pipelines

Inventors: Debarshi Ray (Brno, CZ); Arjun Shankar (Brno, CZ)
Assignee: Red Hat, Inc.
G06T1/20G06F8/4441
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,087,429
App. No.
16/517,155
Granted
Aug 10, 2021
Kind
B2
Abstract

A method comprises receiving, by a media pipeline framework, a plurality of copies of each block of a media pipeline, wherein a first copy of the plurality of copies is a high-level representation of the respective block and wherein the second copy of the plurality of copies is a machine-readable copy. The method further comprises generating, by a processing device, a run-time-optimized media pipeline using the first copy and the second copy.

Claims (29)

1. A method, comprising:

receiving, by a media pipeline framework, a plurality of copies of each block of a media pipeline, wherein a first copy of the plurality of copies is a high-level representation of a respective block, wherein a block comprises discrete processing logic that performs a specified function on media of the media pipeline, wherein the high-level representation comprises descriptions of underlying source-code, and wherein a second copy of the plurality of copies is a machine-readable copy of the respective block, and wherein the high-level representation of the respective block differs from the machine-readable copy of the respective block; and

generating, by a processing device, a run-time-optimized media pipeline using the first copy being the high-level representation and the second copy being the machine readable copy.

2. The method of claim 1 , wherein at least one of the first copy or the second copy comprise an annotation indicating a description of code that runs for each output pixel of an output image.

3. The method of claim 2 , further comprising generating, by a just-in-time (JIT) compiler, the run-time-optimized media pipeline using the annotation.

4. The method of claim 3 , wherein generating the run-time optimized media pipeline comprises combining two or more blocks of the media pipeline based on the annotation.

5. The method of claim 3 , wherein generating the run-time optimized media pipeline comprises eliminating a subset of a block of the media pipeline based on the annotation.

6. The method of claim 3 , wherein generating the run-time optimized media pipeline comprises designating an argument of a block as a constant.

7. The method of claim 3 , wherein generating the run-time optimized media pipeline comprises excluding a subset of a block from optimization.

8. A system, comprising:

a memory to store an input image and an output image; and

a processing device, operatively coupled to the memory, to:

generate a plurality of copies of each block of a media pipeline, wherein a first copy of the plurality of copies is a high-level representation of a respective block,

wherein a block comprises discrete processing logic that performs a specified function on media of the media pipeline, wherein the high-level representation comprises descriptions of underlying source-code, and wherein a second copy of the plurality of copies is a machine-readable copy of the respective block, and wherein the high-level representation of the respective block differs from the machine-readable copy of the respective block; and

provide the plurality of copies of each block to a run-time compiler to generate a run-time-optimized media pipeline to receive the input image and generate the output image, using the first copy being the high-level representation and the second copy being the machine readable copy.

9. The system of claim 8 , wherein at least one of the first copy or the second copy comprise an annotation indicating a description of code that runs for each output pixel of the output image.

10. The system of claim 9 , wherein the run-time compiler is a just-in-time (JIT) compiler, and wherein the annotation instructs the JIT compiler to generate the run-time-optimized media pipeline.

11. The system of claim 10 , wherein the annotation instructs the JIT compiler to combine two or more blocks of the media pipeline.

12. The system of claim 10 , wherein the annotation instructs the JIT compiler to eliminate a subset of a block of the media pipeline.

13. The system of claim 10 , wherein the annotation instructs the JIT compiler to designate an argument of a block as a constant.

14. The system of claim 10 , wherein the annotation instructs the JIT compiler to exclude a subset of a block from optimization.

15. A non-transitory computer-readable storage medium including instructions that, when executed by a processing device of a source host system, cause the processing device to:

receive, by a media pipeline framework, a plurality of copies of each block of a media pipeline, wherein a first copy of the plurality of copies is a high-level representation of a respective block, wherein a block comprises discrete processing logic that performs a specified function on media of the media pipeline, wherein the high-level representation comprises descriptions of underlying source-code, and wherein a second copy of the plurality of copies is a machine-readable copy of the respective block, and wherein the high-level representation of a respective block differs from the machine-readable copy of the respective block; and

generate, by the processing device, a run-time-optimized media pipeline to receive an input image and generate an output image, using the first copy being the high-level representation and the second copy being the machine-readable copy.

16. The non-transitory computer-readable storage medium of claim 15 , wherein at least one of the first copy or the second copy comprise an annotation indicating a description of code that runs for each output pixel of the output image.

17. The non-transitory computer-readable storage medium of claim 16 , the processing device further to generate, by a just-in-time (JIT) compiler, the run-time-optimized media pipeline using the annotation.

18. The non-transitory computer-readable storage medium of claim 16 , wherein to generate the run-time optimized media pipeline the processing device is to combine two or more blocks of the media pipeline based on the annotation.

19. The non-transitory computer-readable storage medium of claim 16 , wherein to generate the run-time optimized media pipeline the processing device is to eliminate a subset of a block of the media pipeline based on the annotation.

20. The non-transitory computer-readable storage medium of claim 16 , wherein to generate the run-time optimized media pipeline the processing device is to designate an argument of a block as a constant.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 19, 2019
From: RAY, DEBARSHI; SHANKAR, ARJUN
To: RED HAT, INC.
Reel/Frame 049807/0752 →
Continuity (1)
Related Publication 20210019857A1 · Jan 21, 2021