IP Library Granted Patent US 8,023,562
Granted Patent B2
US 8,023,562 · App. 12/205,482 · Granted Sep 20, 2011

Real-time video coding/decoding

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,023,562
App. No.
12/205,482
Granted
Sep 20, 2011
Kind
B2
Abstract

A video codec having a modular structure for encoding/decoding a digitized sequence of video frames in a multi-core system is described. The video codec comprises a memory unit; a multithreading engine. and a plurality of control and task modules organized in a tree structure, each module corresponding to a coding operation. The modules communicate with each other by control messages and shared memory. The control modules control all coding logic and workflow, and lower level task modules perform tasks and provide calculations upon receiving messages from the control task modules. The multithreading engine maintains context of each task and assigns at least one core to each task for execution. The method of coding/decoding comprises denoising, core motion estimation, distributed motion estimation, weighted texture prediction and error resilient decoding.

Claims (72)

1. A method of encoding, a digitized sequence of video frames in a multi-core system, the method comprising:

performing core motion estimation;

determining weighted texture prediction use from a difference between normalized density functions for luminance histograms of an original frame and of a reference frame; and

calculating parameters for weighted texture prediction;

wherein the core motion estimation comprises:

dividing a current video frame and a reference video frame into blocks of pixels;

defining in the current frame an initial block B(F, X INIT , Y INIT ) with coordinates X INIT , Y INIT ;

selecting in the reference frame a plurality of N candidate blocks with coordinates (X j , Y j ) where j=0, 1 . . . N, using a predetermined criterion; and

performing core motion estimation for the current frame based on similarity between pixels of a block of the current frame and pixels of blocks of the reference frame;

wherein the motion estimation comprises:

calculating values of a function of distortion Q(F, R, X j , Y j ) for all N candidate blocks; and

comparing minimum values of Q(F, R, X j , Y j ) with predetermined thresholds to find coordinates for motion estimation X j =X MIN , Y j =Y MIN from the function of distortion having a lowest value: Q MIN =Q(F, R, X MIN , Y MIN ).

2. The method of claim 1 , wherein the motion estimation algorithm comprises:

calculating Q(F,R, X MIN +j*Dx MIN ,Y MIN +j*Dy MIN ) for increasing j until inequality (I) is true, or a boundary of the frame is reached:

Q ( F,R,X MIN +j*Dx MIN ,Y MIN +j*Dy MIN )<= Q ( F,R,X MIN +( j+ 1)* Dx MIN ,Y MIN +( j+ 1)* Dy MIN )  (I)

 where j=0, 1, 2 . . . J, and J− is minimum nonnegative integer value for which the inequality (I) is true; Dx and Dy are steps in horizontal and vertical direction;

defining parameters X MIN =X MIN +J*Dx MIN , Y MIN =Y MIN +J*Dy MIN ;

calculating Q MIN =Q(F, R, X MIN , Y MIN );

finding one resulting coordinate pair X MIN , Y MIN for Q MIN =Q(F,R, X MIN ,Y MIN ) from the following coordinate pairs:

(X MIN +Dx MIN ,Y MIN +Dy MIN );

(X MIN −Dx MIN ,Y MIN +Dy MIN );

(X MIN +Dx MIN ,Y MIN −Dy MIN ), and

(X MIN −Dx MIN ,Y MIN −Dy MIN ).

3. A method of encoding a digitized sequence of video frames in a multi-core system, comprising distributed motion estimation for encoding successive frames using one reference frame and the core motion estimation algorithm of claim 2 , wherein the distributed motion estimation comprises:

reducing frame size using reduction coefficient in each spatial dimension;

calculating and storing a plurality of spatially reduced frames;

selecting reduced block size in reduced frames as 8×8;

calculating and storing motion vector sets for reduced frame pairs using the core motion estimation algorithm;

up-scaling the motion vectors for each block according to the reduction coefficients in each dimension and defining a rough full-pel motion vector for each block of an up-scaled frame;

calculating a full-pel motion vector by applying the core motion estimation algorithm to the rough full-pel motion vector as an additional candidate, and reducing search range according to the reduction coefficients; and

calculating a quarter-pel motion vector for each block by applying the core motion estimation algorithm to the full-pel motion vector as a starting point and a search range of 2.

4. A method of encoding a digitized sequence of video frames in a multi-core system, the method comprising:

performing core motion estimation;

determining weighted texture prediction use from a difference between normalized density functions for luminance histograms of an original frame and of a reference frame; and

calculating parameters for weighted texture prediction;

wherein the weighted texture prediction comprises:

performing estimate for normalized density functions for histogram H o of the original frame and histogram H r of the reference frame;

comparing the difference between the density function for H o and the density function H r with a predetermined threshold, and determining that the weighted prediction is to be applied if the difference is less than the threshold; and

calculating weighted prediction parameters A and B for at least one block of pixels P(x, y):

A*L(x+mx,y+my)+B:

A=So/Sr ; and

B=Mo−A*Mr;

where L(x, y) is luminance of the reference frame;

(mx, my) is motion vector for the block P(x, y);

So is deviation for the original frame;

Sr is deviation for the reference frame;

Mo is mean value for the original frame; and

Mr is mean value for the reference frame.

5. A method of encoding a digitized sequence of video frames in a multi-core system, the method comprising:

performing core motion estimation;

determining weighted texture prediction use from a difference between normalized density functions for luminance histograms of an original frame and of a reference frame; and

calculating parameters for weighted texture prediction;

wherein the method further comprises the step of:

pre-processing temporal denoising, including:

providing motion estimation by creating a texture prediction frame for a current frame from a previous frame and a corresponding motion vector;

performing deblocking of the prediction frame; and

modifying the current frame based on at least one pixel of the current frame and at least one pixel of the prediction frame:

I=I +sign( I−P )· F (abs( I−P )),

where:

I is the pixel of the current frame,

P is same position pixel of the prediction frame after deblocking,

abs is the absolute value function,

sign is a sign function (sign(x)=−1, if x<0, sign(x)=0, if x=0, and sign(x)=1, if x>0), and

F is a discrete time function.

6. The method of claim 5 , wherein the discrete time function has the following properties:

F(D) is sufficiently smooth inside interval [0; T 2 ];

F (0)=0;

F ( D )≧ D/ 2 and F ( D )≦ D , if Dε[ 0 ;T 0 );

F ( D )= D/ 2, if Dε[T 0 ;T 1 ); and

F ( D )≦ D/ 2 and F ( D )≧0, if Dε[T 1 ;T 2 );

where T 0 , T 1 and T 2 are predetermined parameters which control strength of noise suppression, 0≦T 0 <T 1 <T 2 .

7. The method according to claim 6 , wherein the de-noising is not performed if a motion vector function mv_cost exceeds a pre-defined threshold.

Assignments (5)
LIEN Recorded Feb 18, 2025
From: SILICON VALLEY BANK
To: BEAMR IMAGING LTD.; BEAMR INC.
Reel/Frame 070244/0396 →
SECURITY INTEREST Recorded Feb 25, 2022
From: BEAMR IMAGING LTD.
To: SILICON VALLEY BANK
Reel/Frame 059097/0195 →
SECURITY INTEREST Recorded Feb 21, 2017
From: BEAMR IMAGING LTD
To: SILICON VALLEY BANK
Reel/Frame 041313/0759 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 28, 2016
From: VANGUARD VIDEO, LLC
To: BEAMR IMAGING LTD.
Reel/Frame 039285/0571 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 25, 2015
From: VANGUARD SOFTWARE SOLUTIONS, INC.
To: VANGUARD VIDEO LLC
Reel/Frame 037138/0180 →