IP Library Granted Patent US 12,155,834
Granted Patent B2
US 12,155,834 · App. 17/500,534 · Granted Nov 26, 2024

Motion compensation and motion estimation leveraging a continuous coordinate system

Inventors: Luca Rossato (Milan, IT); Guido Meardi (Milan, IT)
Assignee: V-Nova International Limited
H04N19/126G06T7/248G06T9/00G06T9/40H04N19/136H04N19/23H04N19/30H04N19/33H04N19/40H04N19/50H04N19/57H04N19/573H04N19/59H04N19/62H04N19/63H04N19/87H04N19/94H04N5/145H04N19/635
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,155,834
App. No.
17/500,534
Granted
Nov 26, 2024
Kind
B2
Abstract

Computer processor hardware receives settings information for a first image. The first image includes a set of multiple display elements. The computer processor hardware receives motion compensation information for a given display element in a second image to be created based at least in part on the first image. The motion compensation information indicates a coordinate location within a particular display element in the first image to which the given display element pertains. The computer processor hardware utilizes the coordinate location as a basis from which to select a grouping of multiple display elements in the first image. The computer processor hardware then generates a setting for the given display element in the second image based on settings of the multiple display elements in the grouping.

Claims (42)

1. A motion compensation system within a signal processor configured as a decoder, the motion compensation system comprising:

a motion compensation block to receive a reference image for a video signal and information on motion compensation and to generate a motion compensated image; and

a combiner to combine the motion compensated image with residual data to reconstruct a rendition of a target image for the video signal,

wherein the information on motion compensation is used to determine a plurality of motion vectors for a plurality of display elements associated with the motion compensated image,

wherein each motion vector is used to determine a set of fractional coordinates, the set of fractional coordinates being independent of an actual sample grid of the reference image and indicating a location within the reference image that is at a resolution that is higher than the reference image,

wherein the set of fractional coordinates are used to determine a grouping of multiple display elements in the reference image, the multiple display elements in the reference image belonging to the actual sample grid of the reference image and having been selected as being near to a position indicated by the set of fractional coordinates which are independent of the actual sample grid of the reference image,

wherein, using on-the-fly resampling, the grouping of multiple display elements in the reference image are weighted to determine values for display elements in the motion compensated image, the respective weights of the multiple display elements determined by the distance of the respective display element from the position indicated by the set of fractional coordinates,

wherein the combiner coverts coordinates of elements in a destination image into homogenous coordinates,

wherein the motion compensated image is generated, at least in part, by applying an affine transformation matrix to the converted homogenous coordinates, and

wherein the set of fractional coordinates indicate which coefficients to use in a filter.

2. The motion compensation system of claim 1 , wherein a set of weights for the grouping of multiple display elements are determined according to a resampling kernel.

3. The motion compensation system of claim 2 , wherein the set of weights are retrieved from a look-up table.

4. The motion compensation system of claim 1 , wherein the plurality of motion vectors provide for the generation of the motion compensated image based on fractional movements of objects from one image plane to another, the fractional movements being at a sub-pixel level.

5. The motion compensation system of claim 1 , wherein the affine transformation performs one or more of scaling, rotation, offset, shearing, and perspective change.

6. A method for decoding images of a video signal comprising:

receiving definitions for at least one grouping of elements and a set of descriptive information on motion;

receiving a reference image for the video signal, the reference image comprising a grid of multiple elements at a given resolution;

computing a motion-compensated prediction for a target image of the video signal using the received reference image, the definitions for the at least one grouping of elements, and the set of descriptive information on motion,

wherein the at least one grouping of elements corresponds to at least one rectangular block of display elements within the target image;

receiving and decoding residual data; and

combining the prediction for the target image with the residual data, to produce a rendition of the target image,

wherein computing a motion-compensated prediction comprises determining a plurality of motion vectors,

wherein each motion vector is used to determine a set of fractional coordinates, the fractional coordinates being independent of an actual sample grid of the reference image, and the fractional coordinates indicating a location distinct from the grid of multiple elements at a given resolution of the reference image,

wherein a setting for a particular display element in the target image is computed on-the-fly and is based on settings of the multiple display elements in the grouping which are weighted, at least in part, on a proximity to a location indicated by the fractional coordinates,

wherein coordinates of elements in the target image are converted into homogenous coordinates,

wherein the motion-compensated prediction applies an affine transformation matrix to the converted homogenous coordinates, and

wherein the set of fractional coordinates indicate which coefficients to use in a filter.

7. The method of claim 6 , wherein the affine transformation comprises one or more of zoom, rotation and offset motion.

8. The method of claim 6 , wherein the at least one grouping of elements comprises a plurality of non-contiguous groupings of elements.

9. Computer-readable hardware storage having instructions stored thereon, the instructions, when carried out by at least one processing device, causes the at least one processing device to perform operations of:

receiving definitions for at least one grouping of elements and a set of descriptive information on motion;

receiving a reference image for a video signal, the reference image comprising a grid of multiple elements at a given resolution;

computing a motion-compensated prediction for a target image of the video signal using the received reference image, the definitions for the at least one grouping of elements, and the set of descriptive information on motion,

wherein the at least one grouping of elements corresponds to at least one rectangular block of display elements within the target image;

receiving and decoding residual data; and

combining the prediction for the target image with the residual data, to produce a rendition of the target image,

wherein computing a motion-compensated prediction comprises determining a plurality of motion vectors,

wherein each motion vector is used to determine a set of fractional coordinates, the fractional coordinates being independent of an actual sample grid of the reference image, and the fractional coordinates indicating a location distinct from the grid of multiple elements at a given resolution of the reference image,

wherein a setting for a particular display element in the target image is computed on-the-fly and is based on settings of the multiple display elements in the grouping which are weighted, at least in part, on a proximity to a location indicated by the fractional coordinates,

wherein coordinates of elements in the target image are converted into homogenous coordinates,

wherein the motion-compensated prediction applies an affine transformation matrix to the converted homogenous coordinates, and

wherein the set of fractional coordinates indicate which coefficients to use in a filter.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2021
From: ROSSATO, LUCA; MEARDI, GUIDO
To: V-NOVA INTERNATIONAL LIMITED
Reel/Frame 057783/0012 →
Priority Claims (1)
WO PCT/EP2013/059886 · May 13, 2013 · international
Continuity (5)
Continuation 15459893 · Mar 15, 2017
Continuation 13893677 · May 14, 2013
Provisional Application 61647426 · May 15, 2012
Provisional Application 61646797 · May 14, 2012
Related Publication 20220038700A1 · Feb 3, 2022