IP Library Granted Patent US 9,661,340
Granted Patent B2
US 9,661,340 · App. 14/027,028 · Granted May 23, 2017

Band separation filtering / inverse filtering for frame packing / unpacking higher resolution chroma sampling formats

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,661,340
App. No.
14/027,028
Granted
May 23, 2017
Kind
B2
Abstract

When packing a video frame of a higher-resolution chroma sampling format such as YUV 4:4:4 into frames of a lower-resolution chroma sampling format such as YUV 4:2:0, a computing device performs wavelet decomposition (or other band separation filtering) on sample values of chroma components of the higher-resolution frame, producing sample values of multiple bands. The device assigns the sample values of the bands to parts of the lower-resolution frames. During corresponding unpacking operations, a computing device assigns parts of the frames of the lower-resolution chroma sampling format to sample values of multiple bands. The device performs wavelet reconstruction (or other inverse band separation filtering) on the sample values of the bands, producing sample values of chroma components of the frame of the higher-resolution chroma sampling format. Band separation filtering can help improve quality of reconstruction when distortion has been introduced during encoding of the chroma components packed into low-resolution frames.

Claims (60)

1. A method comprising:

receiving one or more frames of a higher-resolution chroma sampling format;

packing the one or more frames of the higher-resolution chroma sampling format into one or more frames of a lower-resolution chroma sampling format, wherein the packing includes:

performing band separation filtering on sample values of chroma components of the one or more frames of the higher-resolution chroma sampling format to produce sample values of plural bands; and

assigning the sample values of the plural bands to parts of the one or more frames of the lower-resolution chroma sampling format; and

signaling an indication of type of the band separation filtering, wherein the type of the band separation filtering is selected from multiple band separation filtering schemes.

2. The method of claim 1 further comprising, after the packing:

encoding the one or more frames of the lower-resolution chroma sampling format.

3. The method of claim 1 wherein the band separation filtering is a three-band wavelet decomposition or four-band wavelet decomposition.

4. The method of claim 1 wherein the band separation filtering uses a filter pair with a lowpass filter (“LPF”) and a highpass filter (“HPF”), and wherein the filter pair is a scaled or unscaled one of:

(a) LPF=[1 1]/2 and HPF=[1 −1]/2;

(b) LPF=[−1 1 8 8 1 −1]/16 and HPF=[1 −1]/2;

(c) LPF=[−1 2 6 2 −1]/8 and HPF=[−1 2 −1]/2;

(d) LPF and HPF based on quadrature mirror filters;

(e) LPF and HPF based on polyphase quadrature filters; and

(f) LPF and HPF based on conjugate quadrature filters.

5. The method of claim 4 wherein division operations for the filter pair:

(a) are implemented with arithmetic right shift operations or integer division operations; and/or

(b) include nearest-integer rounding, dithered rounding or no rounding.

6. The method of claim 1 wherein the band separation filtering includes multiple filtering stages, and wherein the multiple filtering stages include a vertical filtering stage and a horizontal filtering stage.

7. The method of claim 6 wherein normalization to compensate for expansion:

(a) occurs after each filtering stage of the multiple filtering stages;

(b) is at least partially deferred for one or more of the multiple filtering stages; or

(c) is at least partially ignored for one or more of the multiple filtering stages, so as to provide scaling of the sample values of the plural bands.

8. The method of claim 1 wherein the band separation filtering uses lifting and/or includes clipping at least some of the sample values of the plural bands.

9. The method of claim 1 further comprising signaling an indication of filtering phase of the band separation filtering.

10. A computing device comprising a processor and memory, wherein the computing device implements a frame unpacker configured to:

receive an indication of type of inverse band separation filtering, wherein the type of the inverse band separation filtering is selected from multiple inverse band separation filtering schemes;

receive one or more frames of a lower-resolution chroma sampling format; and

unpack the one or more frames of the lower-resolution chroma sampling format into one or more frames of a higher-resolution chroma sampling format, wherein the frame unpacker is configured to:

assign parts of the one or more frames of the lower-resolution chroma sampling format to sample values of plural bands; and

perform the inverse band separation filtering on the sample values of the plural bands to produce sample values of chroma components of the one or more frames of the higher-resolution chroma sampling format.

11. The computing device of claim 10 wherein the computing device also implements a video decoder configured to:

decode the one or more frames of the lower-resolution chroma sampling format; and

provide the one or more frames of the lower-resolution chroma sampling format to the frame unpacker.

12. The computing device of claim 10 wherein the inverse band separation filtering is a three-band wavelet reconstruction or four-band wavelet reconstruction.

13. The computing device of claim 10 wherein the inverse band separation filtering uses a filter pair appropriate to invert band separation filtering applied during encoding, wherein the band separation filtering applied during encoding used a filter pair with a lowpass filter (“LPF”) and a highpass filter (“HPF”) that is a scaled or unscaled one of:

(a) LPF=[1 1]/2 and HPF=[1 −1]/2;

(b) LPF=[−1 8 8 1 −1]/16 and HPF=[1 −1]/2;

(c) LPF=[−12 6 2 −1]/8 and HPF=[−1 2 −1]/2;

(d) LPF and HPF based on quadrature mirror filters;

(e) LPF and HPF based on polyphase quadrature filters; and

(f) LPF and HPF based on conjugate quadrature filters.

14. The computing device of claim 10 wherein the inverse band separation filtering uses a filter pair, and wherein division operations for the filter pair:

(a) are implemented with arithmetic right shift operations or integer division operations; and/or

(b) include nearest-integer rounding, dithered rounding or no rounding.

15. The computing device of claim 10 wherein the inverse band separation filtering includes multiple filtering stages, and wherein the multiple filtering stages include a vertical filtering stage and a horizontal filtering stage.

16. The computing device of claim 15 wherein normalization to compensate for expansion:

(a) occurs after each filtering stage of the multiple filtering stages;

(b) is at least partially deferred for one or more of the multiple filtering stages; or

(c) compensates for scaling of the sample values of the plural bands in pre-processing.

17. The computing device of claim 10 wherein the inverse band separation filtering uses lifting and/or includes clipping at least some of the sample values of the plural bands.

18. The computing device of claim 10 wherein the frame unpacker is further configured to receive an indication of filtering phase of the inverse band separation filtering.

19. A computing device comprising a processor and memory, wherein the computing device implements a frame packer configured to:

receive one or more frames of a higher-resolution chroma sampling format;

pack the one or more frames of the higher-resolution chroma sampling format into one or more frames of a lower-resolution chroma sampling format, wherein the higher-resolution chroma sampling format is a 4:4:4 format, 4:2:2 format or 4:2:0 format, wherein the lower-resolution chroma sampling format is a 4:2:2 format, 4:2:0 format or 4:0:0 format lower than the higher-resolution chroma sampling format, and wherein the frame packer is configured to:

perform wavelet decomposition on sample values of chroma components of the one or more frames of the higher-resolution chroma sampling format to produce sample values of plural bands; and

assign the sample values of the plural bands to parts of the one or more frames of the lower-resolution chroma sampling format; and

signal an indication of type of the wavelet decomposition, wherein the type of the wavelet decomposition is selected from multiple wavelet decomposition schemes.

20. The computing device of claim 19 wherein the 4:4:4 format is a YUV 4:4:4 format, the 4:2:2 format is a YUV 4:2:2 format and the 4:2:0 format is a YUV 4:2:0 format.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 9, 2015
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 039025/0454 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 13, 2013
From: SULLIVAN, GARY J.; MALVAR, HENRIQUE SARMENTO; WU, YONGJUN
To: MICROSOFT CORPORATION
Reel/Frame 031206/0527 →