IP Library › Granted Patent US 12,244,818
Granted Patent B2
US 12,244,818 · App. 18/542,997 · Granted Mar 4, 2025

Selective reference block generation without full reference frame generation

Inventors: Yaowu Xu (Saratoga, CA); Bohan Li (Santa Clara, CA); Jingning Han (Santa Clara, CA)
Assignee: GOOGLE LLC
H04N19/139H04N19/105H04N19/172H04N19/537H04N19/573H04N19/577
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,244,818
App. No.
18/542,997
Granted
Mar 4, 2025
Kind
B2
Abstract

A motion vector for a current block of a current frame is decoded from a compressed bitstream. A location of a reference block within an un-generated reference frame is identified. The reference block is generated using a forward reference frame and a backward reference frame without generating the un-generated reference frame. The reference block is generated by identifying an extended reference block by extending the reference block at each boundary of the reference block by a number of pixels related to a filter length of a filter used in sub-pixel interpolation; and generating pixel values of only the extended reference block by performing a projection using the forward reference frame and the backward reference frame without generating the whole of the un-generated reference frame. The current block is then decoded based on the reference block and the motion vector.

Claims (48)

1. A method, comprising:

decoding, from a compressed bitstream, a motion vector for a current block of a current frame;

identifying a location of a reference block within an un-generated reference frame;

generating the reference block using a forward reference frame and a backward reference frame without generating the un-generated reference frame by:

identifying an extended reference block by extending the reference block at each boundary of the reference block by a number of pixels related to a filter length of a filter used in sub-pixel interpolation; and

generating pixel values of only the extended reference block by performing a projection using the forward reference frame and the backward reference frame without generating the whole of the un-generated reference frame; and

decoding the current block based on the reference block and the motion vector.

2. The method of claim 1 , further comprising:

decoding, from the compressed bitstream, a flag indicating that the current block is encoded using the un-generated reference frame.

3. The method of claim 2 , further comprising:

omitting decoding from the compressed bitstream an index of a reference frame to be used for decoding the current block in response to the flag indicating that the current block is encoded using the un-generated reference frame.

4. The method of claim 1 , wherein the un-generated reference frame is temporally co-located with the current frame.

5. The method of claim 1 , further comprising:

determining that motion prediction using the un-generated reference frame is available for the current frame.

6. The method of claim 5 , wherein the motion prediction using the un-generated reference frame is determined to be available in response to determining that a reference frame buffer includes the forward reference frame and the backward reference frame.

7. The method of claim 1 , wherein the number of the pixels is equal to half of the filter length.

8. A device comprising:

a processor configured to:

decode, from a compressed bitstream, a motion vector for a current block of a current frame;

identify a location of a reference block within an un-generated reference frame;

generate the reference block using a forward reference frame and a backward reference frame without generating the un-generated reference frame, wherein to generate the reference block comprises to:

identify an extended reference block by extending the reference block at each boundary of the reference block by a number of pixels related to a filter length of a filter used in sub-pixel interpolation; and

generate pixel values of only the extended reference block by performing a projection using the forward reference frame and the backward reference frame without generating the whole of the un-generated reference frame; and

decode the current block based on the reference block and the motion vector.

9. The device of claim 8 , wherein the processor is further configured to:

decode, from the compressed bitstream, a flag indicating that the current block is encoded using the un-generated reference frame.

10. The device of claim 9 , wherein the processor is further configured to:

omit decoding from the compressed bitstream an index of a reference frame to be used for decoding the current block in response to the flag indicating that the current block is encoded using the un-generated reference frame.

11. The device of claim 8 , wherein the un-generated reference frame is temporally co-located with the current frame.

12. The device of claim 8 , wherein the processor is further configured to:

determine that motion prediction using the un-generated reference frame is available for the current frame.

13. The device of claim 12 , wherein the motion prediction using the un-generated reference frame is determined to be available in response to determining that a reference frame buffer includes the forward reference frame and the backward reference frame.

14. The device of claim 8 , wherein the number of the pixels is equal to half of the filter length.

15. A non-transitory computer-readable storage medium having stored thereon a compressed bitstream, wherein the compressed bitstream is configured for decoding by operations comprising:

decoding, from the compressed bitstream, a motion vector for a current block of a current frame;

identifying a location of a reference block within an un-generated reference frame;

generating the reference block using a forward reference frame and a backward reference frame without generating the un-generated reference frame by:

identifying an extended reference block by extending the reference block at each boundary of the reference block by a number of pixels related to a filter length of a filter used in sub-pixel interpolation; and

generating pixel values of only the extended reference block by performing a projection using the forward reference frame and the backward reference frame without generating the whole of the un-generated reference frame; and

decoding the current block based on the reference block and the motion vector.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the operations further comprise:

decoding, from the compressed bitstream, a flag indicating that the current block is encoded using the un-generated reference frame.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the operations further comprise:

omitting decoding from the compressed bitstream an index of a reference frame to be used for decoding the current block in response to the flag indicating that the current block is encoded using the un-generated reference frame.

18. The non-transitory computer-readable storage medium of claim 15 , wherein the un-generated reference frame is temporally co-located with the current frame.

19. The non-transitory computer-readable storage medium of claim 15 , wherein the operations further comprise:

determining that motion prediction using the un-generated reference frame is available for the current frame.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the motion prediction using the un-generated reference frame is determined to be available in response to determining that a reference frame buffer includes the forward reference frame and the backward reference frame.

Assignments (2)
CORRECTIVE ASSIGNMENT TO CORRECT THE THE MISSING INVENTOR PREVIOUSLY RECORDED ON REEL 65938 FRAME 949. ASSIGNOR(S) HEREBY CONFIRMS THE THE ASSIGNMENT. Recorded Jan 21, 2025
From: XU, YAOWU; LI, BOHAN; HAN, JINGNING
To: GOOGLE LLC
Reel/Frame 069958/0948 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 22, 2023
From: XU, YAOWU; LI, BOHAN
To: GOOGLE LLC
Reel/Frame 065938/0949 →
Continuity (4)
Continuation 17738105 · May 6, 2022
Continuation 16866591 · May 5, 2020
Continuation 15817369 · Nov 20, 2017
Related Publication 20240195979A1 · Jun 13, 2024
References Cited (46)
US 6097854A · Szeliski et al. · 2000 [cited by applicant]
US RE39279E · Yukitake et al. · 2006 [cited by applicant]
US 8184200B1 · Biswas et al. · 2012 [cited by applicant]
US 20040252759A1 · John Winder et al. · 2004 [cited by applicant]
US 20060233253A1 · Shi et al. · 2006 [cited by applicant]
US 20080204592A1 · Jia et al. · 2008 [cited by applicant]
US 20090148058A1 · Dane et al. · 2009 [cited by applicant]
US 20110149106A1 · Kino · 2011 [cited by applicant]
US 20110249734A1 · Segall et al. · 2011 [cited by applicant]
US 20120237114A1 · Park et al. · 2012 [cited by applicant]
US 20130070856A1 · Sato · 2013 [cited by applicant]
US 20130114002A1 · Carlsson et al. · 2013 [cited by applicant]
US 20130121416A1 · He et al. · 2013 [cited by applicant]
US 20140037982A1 · Ivanov et al. · 2014 [cited by applicant]
US 20140307982A1 · Kanaev et al. · 2014 [cited by applicant]
US 20150078456A1 · Hannuksela · 2015 [cited by applicant]
US 20150339806A1 · Wu et al. · 2015 [cited by applicant]
US 20160286232A1 · Li et al. · 2016 [cited by applicant]
US 20170094305A1 · Li · 2017 [cited by examiner]
US 20180376166A1 · Chuang · 2018 [cited by examiner]
US 20190068991A1 · Xu et al. · 2019 [cited by applicant]
US 20200029071A1 · Kang et al. · 2020 [cited by applicant]
JP 2014511054A · 2014 [cited by applicant]
WO 2017133661A1 · 2017 [cited by applicant]
Bankoski et al., “VP8 Data Format and Decoding Guide”, Independent Submission RFC 6389, Nov. 2011, 305 pp. [cited by applicant]
Bankoski et al., “VP8 Data Format and Decoding Guide draft-bankoski-vp8-bitstream-02”, Network Working Group, Internet-Draft, May 18, 2011, 288 pp. [cited by applicant]
“Introduction to Video Coding Part 1: Transform Coding”, Mozilla, Mar. 2012, 171 pp. [cited by applicant]
“Overview VP7 Data Format and Decoder”, Version 1.5, On2 Technologies, Inc., Mar. 28, 2005, 65 pp. [cited by applicant]
Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, Amendment 2: New profiles for professional applications, International Telecommunication Union, Apr. 2007, 75 … [cited by applicant]
Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, Advanced video coding for generic audiovisual services, Amendment 1: Support of additional colour spaces and r… [cited by applicant]
Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, Advanced video coding for generic audiovisual services, Version 3, International Telecommunication Union, Mar.… [cited by applicant]
Bankoski, et al., “Technical Overview of VP8, An Open Source Video Codec for the Web”, Jul. 11, 2011, 6 pp. [cited by applicant]
Series H: Audiovisual and Multimedia Systems, Coding of moving video: Implementors Guide for H.264: Advanced video coding for generic audiovisual services, International Telecommunication Union, Jul. 30, 2010, 15 pp. [cited by applicant]
Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, Advanced video coding for generic audiovisual services, International Telecommunication Union, Version 11, Mar… [cited by applicant]
Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, Advanced video coding for generic audiovisual services, International Telecommunication Union, Version 12, Mar… [cited by applicant]
Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, Advanced video coding for generic audiovisual services, Version 8, International Telecommunication Union, Nov.… [cited by applicant]
Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, Advanced video coding for generic audiovisual services, Version 1, International Telecommunication Union, May … [cited by applicant]
“VP8 Data Format and Decoding Guide, WebM Project”, Google On2, Dec. 1, 2010, 103 pp. [cited by applicant]
“VP6 Bitstream and Decoder Specification”, Version 1.02, On2 Technologies, Inc., Aug. 17, 2006, 88 pp. [cited by applicant]
“VP6 Bitstream and Decoder Specification”, Version 1.03, On2 Technologies, Inc., Oct. 29, 2007, 95 pp. [cited by applicant]
Sun, Deqing et al.; “Learning Optical Flow”; ECCV 2008, Part III, LNC 5304; pp. 83-97. [cited by applicant]
International Search Report and Written Opinion in PCT/US2018/032054, mailed Jul. 23, 2018, 14 pgs. [cited by applicant]
Yi Chin et al., “Dense true motion field compensation for video coding”, 2013 IEEE International Conference on Image Processing, IEEE, (Sep. 15, 2013), pp. 1958-1961. [cited by applicant]
J. Chen et al., “Algorithm description of Joint Exploration Test Model 7 (JEM7)”, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC1/SC29/WG11, 7th Meeting, Torino, Italy, Jul. 13-21, 2017 (url: ht… [cited by applicant]
Alexander Alshin et al., “Bi-directional Optical Flow for Future Video Codec”, 2016 Data Compression Conference (DCC), IEEE, Mar. 30, 2016, pp. 83-90. [cited by applicant]
A. Alshin et al., “Bi-directional optical flow for improving motion compensation”, 2010 Picture Coding Symposium (PCS 2010), Nagoya, Japan, Dec. 8-10, 2010 (IEEE, Piscataway, NJ), pp. 422-425. [cited by applicant]
Cited By (1)
US 12,574,501