IP Library › Granted Patent US 12,432,383
Granted Patent B2
US 12,432,383 · App. 18/542,850 · Granted Sep 30, 2025

Transform prediction with parsing independent coding

Inventors: Onur Guleryuz (San Francisco, CA); Zeyu Deng (Santa Barbara, CA); Debargha Mukherjee (Cupertino, CA); Lester Lu (Los Angeles, CA); Yue Chen (Kirkland, WA)
Assignee: GOOGLE LLC
H04N19/61H04N19/105H04N19/12H04N19/124H04N19/13H04N19/172H04N19/176H04N19/42H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,432,383
App. No.
18/542,850
Granted
Sep 30, 2025
Kind
B2
Abstract

Transform prediction with parsing independent coding includes generating a reconstructed frame and outputting the reconstructed frame. Generating the reconstructed frame includes entropy decoding transform blocks for the reconstructed frame, entropy decoding decoded transform identifiers for the transform blocks, obtaining transform-specific probability distributions for available transforms, and, for a current transform block from the transform blocks, identifying a current remapped transform identifier from the decoded transform identifiers, identifying a current transform identifier in accordance with the current remapped transform identifier, the transform coefficients from the current transform block, and the transform-specific probability distributions, identifying a current transform in accordance with the current transform identifier; inverse transforming, in accordance with the current transform, the current transform block to obtain a current residual block and obtaining a current reconstructed block using the current residual block. Generating the reconstructed frame includes including the current reconstructed block in the reconstructed frame.

Claims (89)

1. A method comprising:

obtaining an encoded bitstream;

generating a reconstructed frame, wherein the reconstructed frame includes a plurality of blocks, wherein generating the reconstructed frame includes:

entropy decoding, from the encoded bitstream, transform blocks for the reconstructed frame;

entropy decoding, from the encoded bitstream, decoded transform identifiers for the transform blocks;

obtaining transform-specific probability distributions for available transforms;

for a current transform block from the transform blocks:

identifying a current remapped transform identifier from the decoded transform identifiers;

identifying a current transform identifier, wherein identifying the current transform identifier includes identifying the current transform identifier in accordance with the current remapped transform identifier, the transform coefficients from the current transform block, and the transform-specific probability distributions;

identifying a current transform in accordance with the current transform identifier;

inverse transforming, in accordance with the current transform, the current transform block to obtain a current residual block; and

obtaining a current reconstructed block using the current residual block; and

including the current reconstructed block in the reconstructed frame; and

outputting the reconstructed frame.

2. The method of claim 1 , wherein entropy decoding the decoded transform identifiers includes entropy decoding the decoded transform identifiers subsequent to entropy decoding the transform blocks.

3. The method of claim 1 , wherein entropy decoding the decoded transform identifiers includes entropy decoding the decoded transform identifiers independent of entropy decoding the transform blocks.

4. The method of claim 1 , wherein identifying the current transform identifier includes inverse remapping the current remapped transform identifier.

5. A method comprising:

obtaining a current frame from an input video stream;

obtaining an encoded frame by encoding the current frame, wherein encoding the current frame includes:

for a current block from the current frame:

obtaining, as a residual block, a difference between the current block and a prediction reference block obtained for coding the current block;

determining a transform identifier of a transform, from a plurality of available transforms, for transforming the residual block;

obtaining an entropy coded block by entropy coding a transform block obtained by transforming the residual block using the transform;

including the entropy coded block in a plurality of entropy coded blocks for the current frame;

obtaining a transform-specific probability distribution trained for the transform;

obtaining an entropy coded transform identifier by entropy coding the transform identifier using the transform-specific probability distribution; and

including the entropy coded transform identifier in a plurality of entropy coded transform identifiers for the current frame;

including the plurality of entropy coded blocks in an output bitstream;

including the plurality of entropy coded transform identifiers in the output bitstream; and

outputting the output bitstream.

6. The method of claim 5 , wherein obtaining the entropy coded block includes:

obtaining quantized transform coefficients by quantizing transform coefficients from the transform block; and

entropy coding the quantized transform coefficients.

7. The method of claim 5 , wherein obtaining the transform-specific probability distribution includes:

generating the transform-specific probability distribution using transform domain training data.

8. The method of claim 5 , wherein obtaining the transform-specific probability distribution includes:

generating the transform-specific probability distribution using pixel domain training data.

9. The method of claim 5 , wherein determining the transform identifier includes determining the transform for transforming the residual block from available transforms, wherein a respective transform identifier is associated with a respective transform from the available transforms.

10. The method of claim 9 , wherein obtaining the transform-specific probability distribution includes obtaining the transform-specific probability distribution in accordance with the transform block and trained statistical probability distributions for the available transforms.

11. The method of claim 10 , wherein obtaining the transform-specific probability distribution includes:

obtaining updated trained statistical probability distributions for the available transforms by updating the trained statistical probability distributions in accordance with the prediction reference block.

12. The method of claim 10 , wherein:

obtaining the entropy coded block includes:

obtaining quantized transform coefficients by quantizing transform coefficients from the transform block; and

entropy coding the quantized transform coefficients; and

obtaining the transform-specific probability distribution includes:

obtaining a plurality of reconstructed residual block candidates by:

for a respective transform from the available transforms:

obtaining a reconstructed residual block candidate by inverse transforming the quantized transform coefficients in accordance with the respective transform; and

including the reconstructed residual block candidate in the plurality of reconstructed residual block candidates; and

obtaining the transform-specific probability distribution output by an artificial neural network using the prediction reference block and the plurality of reconstructed residual block candidates.

13. An apparatus comprising:

a non-transitory computer-readable medium; and

a processor configured to execute instructions stored on the non-transitory computer-readable medium to:

obtain a current frame from an input video stream;

obtain an encoded frame, wherein, to obtain the encoded frame the processor executes the instructions to encode the current frame, wherein to encode the current frame the processor executes the instructions to:

for a current block from the current frame:

obtain as a residual block, a difference between the current block and a prediction reference block obtained for coding the current block;

determine a transform identifier of a transform, from a plurality of available transforms, for transforming the residual block;

obtain an entropy coded block, wherein, to obtain the entropy coded block, the processor executes the instructions to entropy code a transform block obtained by transformation of the residual block using the transform;

include the entropy coded block in a plurality of entropy coded blocks for the current frame;

obtain a transform-specific probability distribution trained for the transform;

obtain an entropy coded transform identifier, wherein to obtain the entropy coded transform identifier the processor executes the instructions to entropy code the transform identifier using the transform-specific probability distribution; and

include the entropy coded transform identifier in a plurality of entropy coded transform identifiers for the current frame;

include the plurality of entropy coded blocks in an output bitstream;

include the plurality of entropy coded transform identifiers in the output bitstream; and

output the output bitstream.

14. The apparatus of claim 13 , wherein obtaining the entropy coded block includes:

obtaining quantized transform coefficients by quantizing transform coefficients from the transform block; and

entropy coding the quantized transform coefficients.

15. The apparatus of claim 13 , wherein obtaining the transform-specific probability distribution includes:

generating the transform-specific probability distribution using transform domain training data.

16. The apparatus of claim 13 , wherein obtaining the transform-specific probability distribution includes:

generating the transform-specific probability distribution using pixel domain training data.

17. The apparatus of claim 13 , wherein determining the transform identifier includes determining the transform for transforming the residual block from available transforms, wherein a respective transform identifier is associated with a respective transform from the available transforms.

18. The apparatus of claim 17 , wherein obtaining the transform-specific probability distribution includes obtaining the transform-specific probability distribution in accordance with the transform block and trained statistical probability distributions for the available transforms.

19. The apparatus of claim 18 , wherein obtaining the transform-specific probability distribution includes:

obtaining updated trained statistical probability distributions for the available transforms by updating the trained statistical probability distributions in accordance with the prediction reference block.

20. The apparatus of claim 18 , wherein:

obtaining the entropy coded block includes:

obtaining quantized transform coefficients by quantizing transform coefficients from the transform block; and

entropy coding the quantized transform coefficients; and

obtaining the transform-specific probability distribution includes:

obtaining a plurality of reconstructed residual block candidates by:

for a respective transform from the available transforms:

obtaining a reconstructed residual block candidate by inverse transforming the quantized transform coefficients in accordance with the respective transform; and

including the reconstructed residual block candidate in the plurality of reconstructed residual block candidates; and

obtaining the transform-specific probability distribution output by an artificial neural network using the prediction reference block and the plurality of reconstructed residual block candidates.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 22, 2023
From: GULERYUZ, ONUR; DENG, ZEYU; MUKHERJEE, DEBARGHA; LU, LESTER; CHEN, YUE
To: GOOGLE LLC
Reel/Frame 065938/0919 →
Continuity (2)
Provisional Application 63433800 · Dec 20, 2022
Related Publication 20240205458A1 · Jun 20, 2024
References Cited (4)
US 9565451B1 · Mukherjee · 2017 [cited by examiner]
US 20180007361A1 · Mukherjee · 2018 [cited by examiner]
US 20180070106A1 · Han · 2018 [cited by examiner]
US 20190058883A1 · Chiang · 2019 [cited by examiner]