IP Library › Granted Patent US 10,645,409
Granted Patent B2
US 10,645,409 · App. 16/018,105 · Granted May 5, 2020

Super-resolution loop restoration

Inventors: Urvang Joshi (Mountain View, CA); Debargha Mukherjee (Cupertino, CA); Andrew Simpson (San Francisco, CA)
Assignee: GOOGLE LLC
H04N19/50H04N7/0117H04N19/117H04N19/176H04N19/184H04N19/619
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,645,409
App. No.
16/018,105
Granted
May 5, 2020
Kind
B2
Abstract

Systems and methods are disclosed for encoding and decoding video. For example, methods may include accessing an encoded bitstream; decoding loop restoration parameters in the encoded bitstream; after reconstruction of an image at a second resolution based on data of the encoded bitstream, upscaling the reconstructed image to obtain an upscaled reconstructed image at a first resolution, wherein the second resolution is less than the first resolution in at least one dimension; and applying loop restoration filtering to the upscaled reconstructed image using the loop restoration parameters to obtain a loop restored image at the first resolution.

Claims (57)

1. A system for encoding video, comprising:

a processor configured to:

access an image at a first resolution;

determine loop restoration parameters based on the image, wherein the loop restoration parameters are associated with loop restoration units of the image;

downscale the image to obtain a downscaled image at a second resolution, wherein the second resolution is less than the first resolution in at least one dimension;

encode the downscaled image in an encoded bitstream, wherein the downscaled image is partitioned into blocks for encoding;

encode the loop restoration parameters in the encoded bitstream, wherein the loop restoration parameters are associated with loop restoration units of the image;

reconstruct the image at the second resolution from the encoded downscaled image;

convert coordinates in the image at the first resolution associated with a loop restoration unit of the loop restoration units to coordinates in the reconstructed image at the second resolution; and

identify a block of the downscaled image based on the converted coordinates, wherein to encode the loop restoration parameters comprises to encode the loop restoration parameters associated with the loop restoration unit in a portion of the encoded bitstream associated with the identified block.

2. The system of claim 1 , wherein the processor is configured to:

upscale the reconstructed image in only a horizontal dimension to obtain an upscaled reconstructed image at the first resolution; and

apply loop restoration filtering to the upscaled reconstructed image at the first resolution.

3. The system of claim 1 , wherein the coordinates correspond to an upper left corner of the loop restoration unit.

4. The system of claim 1 , wherein data from the encoded bitstream associated with one of the blocks includes loop restoration parameters for multiple loop restoration units.

5. The system of claim 1 , wherein the processor is configured to:

upscale the reconstructed image by applying a linear up-scaler to obtain an upscaled reconstructed image at the first resolution; and

apply loop restoration filtering to the upscaled reconstructed image at the first resolution.

6. The system of claim 2 , wherein to apply the loop restoration filtering comprises to apply Weiner filtering using the loop restoration parameters.

7. The system of claim 1 , wherein the processor is configured to:

store or transmit the encoded bitstream.

8. A method for decoding video comprising:

accessing an encoded bitstream;

decoding, from the encoded bitstream, a flag indicating whether an image was encoded using a super-resolution mode;

decoding, responsive to the flag indicating that the image was encoded using a super-resolution mode, bits signaling an amount of downscaling of the image from a first resolution to a second resolution, wherein the second resolution is less than the first resolution in at least one dimension and the amount of downscaling is variable;

decoding loop restoration parameters in the encoded bitstream;

reconstructing the image at the second resolution based on data of the encoded bitstream to obtain a reconstructed image;

upscaling the reconstructed image at the second resolution to obtain an upscaled reconstructed image at the first resolution; and

applying loop restoration filtering to the upscaled reconstructed image using the loop restoration parameters to obtain a loop restored image at the first resolution.

9. The method of claim 8 , wherein the reconstructed image is upscaled in only a horizontal dimension.

10. The method of claim 8 , wherein the loop restoration parameters are associated with loop restoration units of the upscaled reconstructed image at the first resolution and the reconstructed image is partitioned into blocks at the second resolution, the method further comprising:

converting coordinates in the upscaled reconstructed image at the first resolution associated with a loop restoration unit to coordinates in the reconstructed image at the second resolution; and

identifying a block of the reconstructed image based on the converted coordinates, wherein decoding the loop restoration parameters comprises:

decoding loop restoration parameters associated with the loop restoration unit from a portion of the encoded bitstream associated with the identified block.

11. The method of claim 8 , comprising:

storing, transmitting, or displaying an image based on the loop restored image.

12. A system for decoding video, comprising:

a processor configured to:

access an encoded bitstream;

decode loop restoration parameters in the encoded bitstream;

reconstruct an image at a second resolution based on data of the encoded bitstream, wherein the reconstructed image is partitioned into blocks at the second resolution;

upscale the reconstructed image to obtain an upscaled reconstructed image at a first resolution, wherein the second resolution is less than the first resolution in at least one dimension, and the loop restoration parameters are associated with loop restoration units of the upscaled reconstructed image at the first resolution;

convert coordinates in the upscaled reconstructed image at the first resolution associated with a loop restoration unit of the loop restoration units to coordinates in the reconstructed image at the second resolution;

identify a block of the reconstructed image based on the converted coordinates; and

apply loop restoration filtering to the upscaled reconstructed image using the loop restoration parameters to obtain a loop restored image at the first resolution, wherein to decode the loop restoration parameters comprises to decode loop restoration parameters associated with the loop restoration unit from a portion of the encoded bitstream associated with the identified block.

13. The system of claim 12 , wherein the reconstructed image is upscaled in only a horizontal dimension.

14. The system of claim 12 , wherein the coordinates correspond to an upper left corner of the loop restoration unit.

15. The system of claim 12 , wherein data from the encoded bitstream associated with one of the blocks includes loop restoration parameters for multiple loop restoration units.

16. The system of claim 12 , wherein the processor is configured to:

upscale the reconstructed image by applying a linear up-scaler.

17. The system of claim 12 , wherein to apply the loop restoration filtering includes applying Weiner filtering using the loop restoration parameters.

18. The system of claim 12 , wherein the processor is configured to:

store, transmit, or display an image based on the loop restored image.

19. The system of claim 12 , wherein to apply the loop restoration filtering comprises to apply the loop restoration parameters associated with the loop restoration unit to the block of the reconstructed image as upscaled within the upscaled reconstructed image.

20. The system of claim 1 , wherein the processor is configured to:

encode, in the encoded bitstream, a flag indicating that the image was encoded using a super-resolution mode; and

encode, in the encoded bitstream, bits signaling an amount of downscaling of the image from the first resolution to the second resolution, wherein the amount of downscaling is variable.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 27, 2018
From: JOSHI, URVANG; MUKHERJEE, DEBARGHA; SIMPSON, ANDREW
To: GOOGLE LLC
Reel/Frame 046211/0166 →
Continuity (1)
Related Publication 20190394482A1 · Dec 26, 2019