IP Library Granted Patent US 12,413,769
Granted Patent B2
US 12,413,769 · App. 18/544,411 · Granted Sep 9, 2025

Canvas size scalable video coding

Inventors: Taoran Lu (Santa Clara, CA); Fangjun Pu (Sunnyvale, CA); Peng Yin (Ithaca, NY); Sean Thomas McCarthy (San Francisco, CA); Tao Chen (Palo Alto, CA)
Assignee: Dolby Laboratories Licensing Corporation
H04N19/513H04N19/105H04N19/172H04N19/33
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,413,769
App. No.
18/544,411
Granted
Sep 9, 2025
Kind
B2
Abstract

Methods and systems for canvas size scalability across the same or different bitstream layers of a video coded bitstream are described. Offset parameters for a conformance window, a reference region of interest (ROI) in a reference layer, and a current ROI in a current layer are received. The width and height of a current ROI and a reference ROI are computed based on the offset parameters and they are used to generate a width and height scaling factor to be used by a reference picture resampling unit to generate an output picture based on the current ROI and the reference ROI.

Claims (25)

1. An apparatus to decode a coded bitstream, the apparatus comprising:

an input to receive a coded bitstream; and

a processor to process encoded pictures in the coded bitstream, wherein, for a current picture in the coded bitstream, the processor:

receives a current picture width and a current picture height comprising unsigned integer values;

receives first offset parameters determining a rectangular area on the current picture, wherein the first offset parameters comprise signed integer values;

computes a current area width and a current area height for the rectangular area on the current picture based on the current picture width, the current picture height, and the first offset parameters;

for a reference area, accessing a reference area width, a reference area height, a reference area left offset, and a reference area top offset, wherein accessing the reference area width and the reference area height, further comprises, for a reference picture:

accessing a reference picture width and a reference picture height;

receiving second offset parameters determining a rectangular area in the reference picture, wherein the second offset parameters comprise signed integer values;

computing the reference area width and the reference area height for the rectangular area in the reference picture based on the reference picture width, the reference picture height and the second offset parameters; and

computing the reference area left offset and the reference area top offset based on the second offset parameters;

computes a horizontal scaling factor based on the current area width and the reference area width;

computes a vertical scaling factor based on the current area height and the reference area height;

computes a left-offset adjustment and a top-offset adjustment of the current area based on the first offset parameters; and

performs motion compensation based on the horizontal and vertical scaling factors, the left-offset adjustment, the top-offset adjustment, the reference area left offset, and the reference area top offset.

2. The apparatus of claim 1 , wherein the first offset parameters comprise a left offset, a top offset, a right offset, and a bottom offset.

3. The apparatus of claim 2 , wherein one or more of the left offset, the top offset, the right offset, or the bottom offset comprise values between −2 14 and 2 14 .

4. The apparatus of claim 2 , wherein computing the current area width comprises subtracting from the current picture width a first sum of the left offset and the right offset, and computing the current area height comprises subtracting from the current picture height a second sum of the top offset and the bottom offset.

5. The apparatus of claim 1 , wherein computing the horizontal scaling factor (hori_scale_fp) comprises computing hori_scale_fp=((fRefWidth <<14)+ (fCurWidth>>1))/fCurWidth wherein fRefWidth denotes the reference area width and fCurWidth denotes the current area width.

6. The apparatus of claim 1 , wherein performing motion compensation comprises computing

refxSb L =(((xSb-fCurLeftOffset)<<4)+refMvLX [0]) * hori_scale_fp

refx L =((Sign (refxSb L ) *((Abs (refxSb L )+128) >>8) +XL * ((hori_scale_fp+8) >>4))+32) >>6+ (fRefLeftOffset <<4)

refySbL=(((ySb-fCurTopOffset)<<4)+refMvLX [1]) *vert_scale_fp

refyL=((Sign (refySbL) *((Abs (refySbL)+128) >>8)+yL ((vert_scale_fp+8) >>4))+32) >>6+ (fRefTopOffset <<4)

wherein hori_scale_fp denotes the horizontal scaling factor, vert_scale_fp denotes the vertical scaling fcator, fCurLeftOffset denotes the left-offset adjustment, fCurTopOffset denotes the top-offset adjustment, fRefLeftOffset denotes the reference area left offset, fRefTopOffset denotes the reference area top offset, and (refxSb L , refySb L ) and (refx L , refy L ) are luma locations pointed to by a motion vector (refMvLX [0], refMvLX [1]) given in 1/16-sample units.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 1, 2024
From: LU, TAORAN; PU, FANGJUN; YIN, PENG; MCCARTHY, SEAN THOMAS; CHEN, TAO
To: DOLBY LABORATORIES LICENSING CORPORATION
Reel/Frame 066331/0404 →
Continuity (5)
Continuation 17629093
Provisional Application 62945931 · Dec 10, 2019
Provisional Application 62902818 · Sep 19, 2019
Provisional Application 62883195 · Aug 6, 2019
Related Publication 20240121424A1 · Apr 11, 2024
References Cited (62)
US 9554132B2 · Crenshaw · 2017 [cited by applicant]
US 10063886B2 · Ye · 2018 [cited by applicant]
US 10110890B2 · Haque · 2018 [cited by applicant]
US 10116952B2 · Lin · 2018 [cited by applicant]
US 10148983B2 · Hendry · 2018 [cited by applicant]
US 10999585B2 · Atkins · 2021 [cited by applicant]
US 20130034170A1 · Chen · 2013 [cited by applicant]
US 20140192901A1 · Wang · 2014 [cited by applicant]
US 20140328398A1 · Chen · 2014 [cited by applicant]
US 20150195573A1 · Aflaki Beni · 2015 [cited by applicant]
US 20150201198A1 · Marlatt · 2015 [cited by applicant]
US 20150312580A1 · Hannuksela · 2015 [cited by examiner]
US 20160073118A1 · Iguchi · 2016 [cited by applicant]
US 20160073119A1 · Toma · 2016 [cited by applicant]
US 20160100187A1 · Lee · 2016 [cited by applicant]
US 20160165276A1 · Toma · 2016 [cited by applicant]
US 20160191932A1 · Toma · 2016 [cited by applicant]
US 20160234500A1 · Tsukagoshi · 2016 [cited by applicant]
US 20160295257A1 · Iguchi · 2016 [cited by applicant]
US 20170034532A1 · Yamamoto · 2017 [cited by examiner]
US 20170085917A1 · Hannuksela · 2017 [cited by applicant]
US 20170105015A1 · Haque · 2017 [cited by applicant]
US 20170105019A1 · Redmann · 2017 [cited by applicant]
US 20170127070A1 · Suzuki · 2017 [cited by applicant]
US 20170264861A1 · Xu · 2017 [cited by applicant]
US 20170359611A1 · Iguchi · 2017 [cited by applicant]
US 20180063542A1 · Haque · 2018 [cited by applicant]
US 20180098079A1 · Chuang · 2018 [cited by applicant]
US 20180146235A1 · Tsukagoshi · 2018 [cited by applicant]
US 20180166102A1 · Newman · 2018 [cited by applicant]
US 20190045116A1 · Yamamoto · 2019 [cited by applicant]
CN 104813665A · 2015 [cited by applicant]
CN 105744251A · 2016 [cited by applicant]
CN 107153876A · 2017 [cited by applicant]
CN 105359526B · 2019 [cited by applicant]
EP 3120562B1 · 2020 [cited by applicant]
JP 2016506691A · 2016 [cited by applicant]
JP 2016226035A · 2016 [cited by applicant]
JP 2017005723A · 2017 [cited by applicant]
JP 2018534827A · 2018 [cited by applicant]
WO 2015053287A1 · 2015 [cited by applicant]
WO 2015115946A1 · 2015 [cited by applicant]
WO 2018123542A1 · 2018 [cited by applicant]
WO 2019067762A1 · 2019 [cited by applicant]
Alshina et al: “Re-sampling process modification considering cropping information”, JCT-VC Meeting; Mar. 27, 2014-Apr. 4, 2014; Valencia; (Joint Collaborative Team on Video Coding of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.… [cited by applicant]
Brendel, H. et al “What is more Compelling: More Pixels or Higher Frame Rate?” SMPTE 2012, pp. 1-50. [cited by applicant]
Bross B et al: “Versatile Video Coding (Draft 6)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothernburg, SE, Jul. 3-12, 2019, Document: JVET-O2001-v3, pp. 1-421. [cited by applicant]
Bross, B. et al “Versatile Video Coding (Draft 4),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, Document: JVET-M1001-v54, pp. 1-287. [cited by applicant]
Carbonara, C. et al “High Frame Rate Capture and Production” SMPTE Meeting Presentation, 2015 Society of Motion Picture & Television Engineers, pp. 1-19. [cited by applicant]
Infrastructure of audiovisual services—Transmission multiplexing and synchronization, H.222.0, Series H, Generic coding of moving pictures and associated audio information: Systems, ITU, Aug. 2018, pp. 1-306. [cited by applicant]
ITU H.265 “High Efficiency Video Coding” Series of moving Video, ITU Feb. 2018. [cited by applicant]
Boyce et al, “Overview of SHVC: Scalable Extensions of the High Efficiency Video Coding (HEVC) Standard”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, No. 1. Jan. 1, 2015, pp. 20-34, 15 pages. [cited by applicant]
Mackin et al “A Frame Rate Conversion Method Based on a Virtual Shutter Angle” IEEE International Conference on Image Processing, Sep. 22, 2019, pp. 4215-4219, 5 pages. [cited by applicant]
McCarthy, S. et al “AHG7: Indication of Shutter Angle for Variable Frame Rate Application” Joint Collaborative Team on Video Coding (JCT-VC) ofITU-T SG 16 WP 3 and 1S0/IEC JTC 1/SC 29/WVG 11, 36th Meeting: Gothenburg, S… [cited by applicant]
Mitchell, Nick “Digital Cinema Mastering” Considerations for Higher Frame Rates SMPTE 2012, pp. 1-48. [cited by applicant]
Omori, Y. et al “A 120 fps High Frame Rate Real-Time HEVC Video Encoder with Parallel Configuration Scalable to 1K” IEEE Transactions on Multi-Scale Computing Systems, vol. 4, No. 4, Oct.-Dec. 2018, pp. 491-499, 9 pages. [cited by applicant]
Skupin et al., “AHG12: On filtering of independently coded region,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, Document: JVET-O0494-… [cited by applicant]
Sullivan, A. et al “On Frame Rate Support and Extraction in VVC” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, Document: JVET-M_Notes_d7… [cited by applicant]
Sullivan G. et al. “Meeting report of the 36th meeting of the Joint Collaborative Team on Video Coding.” In: Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11; 37th Meeting: … [cited by applicant]
Text of ISO/IEC CD 23090-3, ISO/IEC JTC 1/SC 29 /WG 11 N 18692, 27, MPEG Meeting; Jul. 12, 2019, Text of ISO/IEC CD 23090-3, ISO/IEC JTC 1/SC 29/WG 11, pp. 1-105. [cited by applicant]
Ugur, K. et al “Exif Data SEI Message”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 10th Meeting: Stockholm, SE, Jul. 11-20, 2012, Document: Document: JCTVC-J_Not… [cited by applicant]
Wenger et al, “AHG8: Spatial Scalability using Reference Picture Resampling,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, Document: J… [cited by applicant]