IP Library Granted Patent US 9,860,554
Granted Patent B2
US 9,860,554 · App. 14/977,839 · Granted Jan 2, 2018

Motion estimation for uncovered frame regions

Inventors: Jonatan Samuelsson (Stockholm, SE); Kenneth Andersson (Gävle, SE); Clinton Priddle (Carindale, AU)
Assignee: Telefonaktiebolaget LM Ericsson (Publ)
H04N19/527G06T7/20H04N19/132H04N19/172H04N19/513H04N19/553H04N19/587
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,860,554
App. No.
14/977,839
Granted
Jan 2, 2018
Kind
B2
Abstract

In a motion estimation for a group of at least one image element in a frame of a video sequence, a global motion is determined between the frame and a reference frame. Uncovered groups present in an uncovered region of the frame are identified based on the determined global motion. The global motion is assigned as motion representation for these identified uncovered groups. The assigned motion representation is useful for constructing new frames in the sequence in a frame rate up-conversion.

Claims (29)

1. A method of motion estimation for a group of at least one image element in a frame associated with a time instance in a video sequence, said method comprising the steps of:

determining a representation of a global motion of image element property values from at least a portion of a previous frame associated with a previous time instance in said video sequence to at least a portion of said frame, wherein the determining the representation of the global motion comprises estimating elements of a matrix A and a vector b based on a motion set, and wherein the motion set comprises, for each group in said at least a portion of said frame, a respective motion representation referring to an associated group in said previous frame, and wherein the global motion is represented as a vector v=Ax+b, wherein v=[v x y y ] T is the global motion representation, v x is a first vector component of said global motion representation in a first direction, v y is a second vector component of said global motion representation in a second perpendicular direction, and x=[x y] T is an image element position in said frame;

identifying an uncovered group of at least one image element in an uncovered region of said frame based on said global motion representation; and

assigning said global motion representation to said uncovered group as motion estimation for said uncovered group.

2. The method according to claim 1 , wherein said identifying step comprises identifying said uncovered group as a group in said frame that does not have an associated group in said previous frame when applying said global motion from said group in said frame to said previous frame.

3. The method according to claim 1 , further comprising providing the motion set comprising, for each group in said at least a portion of said frame, the respective motion representation referring to the associated group in said previous frame, wherein said determining step comprises determining, based on at least a portion of the motion representations of said motion set, said global motion representation.

4. The method according to claim 3 , wherein said determining step comprises determining, based on said at least a portion of the motion representations of said motion set, a position-dependent global motion representation.

5. The method according to claim 1 , further comprising the steps of:

comparing, for said uncovered group having a neighboring set of neighboring groups not identified as belonging to said uncovered region, motion representations associated with said neighboring groups of said neighboring set with said global motion representation; and

assigning, to said uncovered group, a motion representation provided based on a motion representation associated with a neighboring group of said neighboring set if at least a minimum number of said motion representations differ from said global motion representation with at least a minimum difference.

6. The method according to claim 5 , further comprising comparing said motion representations differing from said global motion representation with at least said minimum difference, wherein said assigning step comprises assigning, to said uncovered group, said motion representation provided based on said motion representation associated with said neighboring group of said neighboring set if said motion representations differing from said global motion representation with at least said minimum difference differ from each other with no more than a maximum difference.

7. The method according to claim 5 , further comprising the steps of:

identifying at least one uncovered group present on a same group row or group column in said frame as said uncovered group but not having any neighboring groups not identified as corresponding to said uncovered region; and

assigning, to said at least one uncovered group, a motion representation assigned to said uncovered group of said same group row or group column if said motion representation assigned to said uncovered group of said same group row or group column is different from said global motion representation.

8. A device for motion estimation for a group of at least one image element in a frame associated with a time instance in a video sequence, said device comprises:

a processor; and

a memory containing instructions executable by said processor, whereby said device is operative to:

determine a representation of a global motion of image element property values from at least a portion of a previous frame associated with a previous time instance in said video sequence to at least a portion of said frame, wherein the determining the representation of the global motion comprises estimating elements of a matrix A and a vector b based on a motion set, and wherein the motion set comprises, for each group in said at least a portion of said frame, a respective motion representation referring to an associated group in said previous frame, and wherein the global motion is represented as a vector v=Ax+b, wherein v=[v x y y ] T is the global motion representation, v x is a first vector component of said global motion representation in a first direction, v y is a second vector component of said global motion representation in a second perpendicular direction, and x=[x y] T is an image element position in said frame;

identify an uncovered group of at least one image element in an uncovered region of said frame based on said global motion representation; and

assign said global motion representation to said uncovered group.

9. The device according to claim 8 , wherein said device is further operative to identify said uncovered group as a group in said frame that does not have an associated group in said previous frame when applying said global motion from said group in said frame to said previous frame.

10. The device according to claim 8 , wherein said device is further operative to provide the motion set comprising, for each group in said at least a portion of said frame, the respective motion representation referring to the associated group in said previous frame, wherein said device is further operative to determine, based on at least a portion of the motion representations of said motion set, said global motion representation.

11. The device according to claim 8 , wherein said device is further operative to compare, for said uncovered group having a neighboring set of neighboring groups not identified as belonging to said uncovered region, motion representations associated with said neighboring groups of said neighboring set with said global motion representation, wherein said device is further operative to assign, to said uncovered group, a motion representation provided based on a motion representation associated with a neighboring group of said neighboring set if at least a minimum number of said motion representations differ from said global motion representation with at least a minimum difference.

12. The device according to claim 11 , wherein

said device is further operative to compare said motion representations differing from said global motion representation with at least said minimum difference, and

said device is further operative to assign, to said uncovered group, said motion representation provided based on said motion representation associated with said neighboring group of said neighboring set if said motion representations differing from said global motion representation with at least said minimum difference differ from each other with no more than a maximum difference.

13. The device according to claim 11 , wherein

said device is further operative to identify at least one uncovered group present on a same group row or group column in said frame as said uncovered group but not having any neighboring groups not identified as corresponding to said uncovered region, and

said device is further operative to assign, to said at least one uncovered group, a motion representation assigned to said uncovered group of said same group row or group column if said motion representation assigned to said uncovered group of said same group row or group column is different from said global motion representation.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 11, 2016
From: SAMUELSSON, JONATAN; ANDERSSON, KENNETH; PRIDDLE, CLINTON
To: TELEFONAKTIEBOLAGET L M ERICSSON (PUBL)
Reel/Frame 037451/0796 →
Continuity (3)
Continuation 12524281
Provisional Application 60897511 · Jan 26, 2007
Related Publication 20160112717A1 · Apr 21, 2016