IP Library Granted Patent US 8,831,101
Granted Patent B2
US 8,831,101 · App. 12/185,077 · Granted Sep 9, 2014

Method and system for determining a metric for comparing image blocks in motion compensated video coding

Inventor: Steven Pigeon (Blainville, CA)
Assignee: Ecole de Technologie Superieure
H04N19/00781H04N19/00593H04N19/00303H04N19/00509H04N19/00145
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,831,101
App. No.
12/185,077
Granted
Sep 9, 2014
Kind
B2
Abstract

Method and system for determination of a metric measuring a difference between two image blocks used in motion compensated video coding of scenes are described. Only selected pixels in a block in the scene are processed for enhancing the speed of the metric computation.

Claims (71)

1. A method for determining a metric for use in motion estimation in motion compensated video coding of a scene comprising frames, the method comprising:

employing at least one Single Instruction Multiple Data (SIMD) processor for determining the metric, comparing a first image block contained in a reference frame and a second image block contained in a current frame, comprising:

generating a pattern matrix having the same number of rows and columns as the first and second image blocks, said pattern matrix specifying a subset of pixels to be selected in the first and second image blocks;

the subset of pixels being divided into a plurality of groups of successive pixels, each group of successive pixels comprising an uninterrupted sequence of two or more pixels in a row, a number of pixels in said each group of successive pixels being selected so that to be operable in a single instruction of the SIMD processor;

positions of the groups of successive pixels within the first and second image blocks being selected so as to maximize a span of block coverage, wherein:

vertical distances between any two adjacent groups are substantially equal; and

horizontal distances between any two adjacent groups are also substantially equal;

obtaining characteristics of pixels in the subset of pixels for the first and second images blocks; and

determining the metric based on the obtained characteristics.

2. The method of claim 1 , wherein the step of generating further comprises:

generating a global motion characteristic of the scene indicating a direction of movement of an object in the scene; and

generating the pattern matrix corresponding to the SIMD processor architecture used for the motion compensated video coding for the global motion characteristic.

3. The method of claim 2 , wherein the step of generating the global motion characteristic further comprises:

determining an average motion vector for the scene, the average motion vector having an amplitude and a direction;

comparing the amplitude of the average motion vector with a given threshold of no movement, comprising:

determining the global motion characteristic as static signifying no object movement when the amplitude of the average motion vector is below the threshold; and

determining the global motion characteristic as one of horizontal movement, vertical movement, or diagonal movement based on the direction of the average motion vector when the amplitude of the average motion vector is equal to or above the threshold.

4. The method of claim 2 , wherein the generating the pattern matrix corresponding to the SIMD processor architecture further comprises selecting the pattern matrix from a series of predetermined pattern matrices based on the SIMD processor architecture used for the motion compensated video coding and the global motion characteristic.

5. The method of claim 4 , further comprising generating a repository of pattern matrices prior to performing motion compensation, each pattern matrix in the repository corresponding to the SIMD processor architecture used for the motion compensated video coding and the global motion characteristic.

6. The method of claim 5 , wherein the generating the repository of pattern matrices further comprises:

for each pattern matrix in the repository of pattern matrices:

storing boolean values in cells of the pattern matrix, each cell corresponding to pixels occupying the same position in the first and second image blocks; and storing one of the “true” or “false” boolean values in those cells of the pattern matrix, which correspond to the pixels from the first and second image blocks to be included in a computation of the metric, and storing the other boolean value in those cells of the pattern matrix, which correspond to the pixels in the first and second image blocks to be excluded from the computation of the metric.

7. The method of claim 1 , wherein the step of obtaining comprises:

obtaining the characteristic, which is a luminance value;

storing luminance values of pixels in the first image block in cells in a first matrix; and

storing luminance values of pixels in the second image block in cells in a second matrix.

8. The method of claim 7 , wherein the processing further comprises:

selecting cells in the first and the second matrices using the pattern matrix; and

applying a Sum of Absolute Differences function on the luminance values stored in the selected cells.

9. The method of claim 7 , wherein the processing further comprises:

selecting cells in the first and the second matrices using the pattern matrix; and

applying a Mean Squared Error function on the luminance values stored in the selected cells.

10. A system for determining a metric for use in motion estimation in motion compensated video coding of a scene comprising frames, the system comprising:

a Single Instruction Multiple Data (SIMD) processor;

a memory comprising computer-readable instructions stored thereon for execution by the SIMD processor, forming:

a metric determination unit, determining the metric comparing a first image block contained in a reference frame and a second image block contained in a current frame, comprising:

a pattern matrix generation unit, generating a pattern matrix, having the same number of rows and columns as the first and second image blocks, the pattern matrix specifying a subset of pixels to be selected in the first and second image blocks;

the subset of pixels being divided into a plurality of groups of successive pixels, each group of successive pixels comprising an uninterrupted sequence of two or more pixels in a row, a number of pixels in each group of successive pixels being selected so that to be operable in a single instruction of the SIMD processor;

positions of the groups of successive pixels within the first and second image blocks being selected so as to maximize a span of block coverage, wherein:

vertical distances between any two adjacent groups are substantially equal; and

horizontal distances between any two adjacent groups are also substantially equal;

an image retrieval unit, obtaining characteristics of pixels for the first and second image blocks; and

a metric computation unit, determining the metric based on the obtained characteristics.

11. The system of claim 10 , wherein the pattern matrix generation unit further comprises a global motion characteristic determination unit, computing a global motion characteristic of the scene indicating a direction of movement of an object in the scene; and wherein the pattern matrix generation unit is further configured to generate the pattern matrix further based on the global motion characteristic of the scene and the SIMD processor architecture used in the motion compensated video coding.

12. The system of claim 11 , wherein the pattern matrix generation unit further comprises:

a pattern matrix repository, storing a series of predetermined pattern matrices, each pattern matrix in the pattern matrix repository corresponding to the SIMD processor architecture used in the motion compensated video coding and the global motion characteristic of the scene; and

a pattern matrix selection unit, selecting the pattern matrix from the pattern matrix repository.

13. The system of claim 10 , wherein the characteristic is a luminance value.

14. The system of claim 13 , wherein the metric computation unit is configured to apply a Sum of Absolute Differences function on the luminance values of pixels in the subset of pixels.

15. The system of claim 13 , wherein the metric computation unit is configured to apply a Mean Squared Error function on the luminance values of pixels in the subset of pixels.

16. A non-transitory computer readable storage medium, having computer readable instructions stored thereon for execution by a Single Instruction Multiple Data (SIMD) processor, causing the processor to perform the steps of the method as described in claim 1 .

17. The method of claim 1 , wherein the number of pixels in the group of successive pixels to be operable in the single instruction of the SIMD processor is equal to 8.

18. The method of claim 1 , further comprising selecting the pattern matrix from a series of predetermined pattern matrices.

19. A system for determining a metric for use in motion estimation in motion compensated video coding of a scene comprising frames, the system comprising:

a Single Instruction Multiple Data (SIMD) processor and a non-transitory computer readable storage medium having computer readable instructions stored thereon for execution by the SIMD processor, causing the SIMD processor to determine the metric comparing a first image block contained in a reference frame and a second image block contained in a current frame, the first and second image blocks having the same number of rows and columns of pixels, comprising:

generating a pattern matrix having the same number of rows and columns as the first and second image blocks, said pattern matrix specifying a subset of pixels to be selected in the first and second image blocks; the subset of pixels being divided into a plurality of groups of successive pixels, each group of successive pixels comprising an uninterrupted sequence of two or more pixels in a row, a number of pixels in each group of successive pixels being selected so that to be operable in a single instruction of the SIMD processor; and positions of the groups of successive pixels within the first and second image blocks being selected so as to maximize a span of block coverage, wherein:

vertical distances between any two adjacent groups are substantially equal; and

horizontal distances between any two adjacent groups are also substantially equal;

obtaining characteristics of pixels in the subset of pixels for the first and second image blocks; and

determining the metric based on the obtained characteristics.

20. The system of claim 19 , wherein the computer-readable instructions further cause the SIMD processor to select the pattern matrix from a series of predetermined pattern matrices.

21. The system of claim 20 , wherein the computer-readable instructions further cause the SIMD processor to:

generate a global motion characteristic of the scene indicating a direction of movement of an object in the scene; and

generate the pattern matrix corresponding to the SIMD processor architecture used for the motion compensated video coding for the global motion characteristic.

22. The system of claim 21 , wherein the computer-readable instructions further cause the SIMD processor to:

determine an average motion vector for the scene, the average motion vector having an amplitude and a direction;

compare the amplitude of the average motion vector with a given threshold of no movement, comprising:

determining the global motion characteristic as static signifying no object movement when the amplitude of the average motion vector is below the threshold; and

determining the global motion characteristic as one of horizontal movement, vertical movement, or diagonal movement based on the direction of the average motion vector when the amplitude of the average motion vector is equal to or above the threshold.

23. The system of claim 21 , wherein the computer-readable instructions further cause the SIMD processor to select the pattern matrix from the series of predetermined pattern matrices based on the SIMD processor architecture used for the motion compensated video coding and the global motion characteristic.

24. The system of claim 23 , wherein the computer-readable instructions further cause the SIMD processor to generate a repository of pattern matrices prior to performing motion compensation, each pattern matrix in the repository corresponding to the SIMD processor architecture used for the motion compensated video coding and the global motion characteristic.

Assignments (6)
RELEASE OF SECURITY INTEREST Recorded Mar 21, 2017
From: NXT CAPITAL LLC
To: VANTRIX CORPORATION
Reel/Frame 042050/0722 →
RELEASE OF SECURITY INTEREST Recorded Jul 15, 2013
From: NXT CAPITAL, LLC
To: VANTRIX CORPORATION
Reel/Frame 030797/0445 →
RELEASE OF SECURITY INTEREST Recorded Jun 14, 2013
From: WF FUND III LIMITED PARTNERSHIP, C.O.B. WELLINGTON FINANCIAL FUND III AND WELLINGTON FINANCIAL LP
To: VANTRIX CORPORATION
Reel/Frame 030615/0063 →
SECURITY AGREEMENT Recorded Mar 20, 2013
From: VANTRIX CORPORATION
To: NXT CAPITAL, LLC, ITS SUCCESSORS AND ASSIGNS
Reel/Frame 030053/0824 →
SECURITY AGREEMENT Recorded Mar 7, 2011
From: VANTRIX CORPORATION
To: WF FUND III LIMITED PARTNERSHIP, C.O.B. WELLINGTON FINANCIAL FUND III AND WELLINGTON FINANCIAL LP
Reel/Frame 025907/0804 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 14, 2008
From: PIGEON, STEVEN
To: ECOLE DE TECHNOLOGIE SUPERIEURE
Reel/Frame 021682/0360 →
Continuity (1)
Related Publication 20100027662A1 · Feb 4, 2010