Granted Patent B2

US 12,506,894 · App. 18/687,768 · Granted Dec 23, 2025

Inventors: Peng Yin (Ithaca, NY); Fangjun Pu (Sunnyvale, CA); Taoran Lu (Santa Clara, CA); Arjun Arora (Sunnyvale, CA); Guan-Ming Su (Fremont, CA); Tao Chen (Palo Alto, CA); Sean Thomas McCarthy (San Francisco, CA); Walter J. Husak (Simi Valley, CA)

Assignee: DOLBY LABORATORIES LICENSING CORPORATION

H04N19/517H04N19/117H04N19/82

Abstract

Claims (24)

1 . A method comprising:

receiving, from an input video signal, an input image represented in an input domain;

performing forward reshaping on the input image to generate a forward reshaped image represented in a reshaped image domain;

wherein the forward reshaped image is generated by performing the forward reshaping with a neural network that includes a non-linear mapping of input codewords in the input image to forward reshaped codewords in N channels, where N represents an integer no less than three;

performing non-reshaping encoding operations to encode the reshaped image into an encoded video signal, wherein at least one of the non-reshaping encoding operations is implemented with a machine learning (ML) model that has been previously trained with training images in one or more training datasets in a preceding training stage;

causing a recipient device of the encoded video signal to generate a reconstructed image from the forward reshaped image, wherein the reconstructed image is used to derive a display image to be rendered on an image display operating with the recipient device.

2 . The method of claim 1 , wherein the neural network represents a first convolutional neural network that uses a convolutional filter of spatial kernel size of 1 pixel×1 pixel to forward reshape each input codeword in the input image in three color channels to a respective forward reshaped codeword in N channels, where N represents an integer no less than three; wherein the reconstructed image is generated by inverse reshaping performed with a second convolutional neural network that uses a second convolutional filter of spatial kernel size of 1 pixel×1 pixel to inverse reshape each forward reshaped codeword in the input image in the N channels to a respective reconstructed codeword in the three color channels.

3 . The method of claim 1 , wherein the non-reshaping encoding operations include one or more of: optical flow analysis, motion vector encoding, motion vector decoding, motion vector quantization, motion compensation, residual encoding, residual decoding, or residual quantization.

4 . The method of claim 1 , wherein the forward reshaping is performed as out-of-loop image processing operations performed before the non-reshaping encoding operations.

5 . The method of claim 1 , wherein the forward reshaping is performed as a part of overall in-loop image processing operations that include the non-reshaping encoding operations.

6 . The method of claim 5 , wherein the overall in-loop image processing operations are encoding operations.

7 . The method of claim 1 , wherein an image metadata portion for the forward reshaped image is a part of image metadata carried by the encoded video signal; wherein the image metadata portion includes one or more of: forward reshaping parameters for the forward reshaping, or backward reshaping parameters for inverse reshaping.

8 . The method of claim 7 , wherein the image metadata portion includes reshaping parameters that explicitly specifies a reshaping mapping for one of the forward reshaping or the inverse reshaping.

9 . The method of claim 8 , wherein the reshaping parameters that explicitly specifies are reshaping mapping are generated by one of: a ML-based reshaping mapping prediction method, or a non-ML-based reshaping mapping generation method.

10 . The method of claim 1 , wherein the image metadata portion includes a reshaping parameter that identifies the forward reshaping as one of: global mapping or image adaptive mapping.

11 . The method of claim 1 , wherein the forward reshaping is performed with an implicit reshaping mapping embodied with weights and biases of a neural network that have been previously trained with training images in one or more training datasets.

12 . A method comprising:

decoding, from an encoded video signal, a forward reshaped image represented in a reshaped image domain, wherein the forward reshaped image was generated by an upstream device by forward reshaping an input image represented in an input image domain;

wherein the forward reshaped image was generated by the upstream device performing the forward reshaping with a neural network that includes a non-linear mapping of input codewords in the input image to forward reshaped codewords in N channels, where N represents an integer no less than three;

performing inverse reshaping on, as well as non-reshaping decoding operations in connection with, the forward reshaped image to generate a reconstructed image represented in a reconstructed image domain, wherein the inverse reshaping and forward reshaping form a reshaping operation pair, wherein at least one of the non-reshaping decoding operations is implemented with a machine learning (ML) model that has been previously trained with training images in one or more training datasets in a preceding training stage;

causing a display image derived from the reconstructed image to be rendered on an image display.

13 . The method of claim 12 , wherein the inverse reshaping is performed with an implicit reshaping mapping embodied with weights and biases of a neural network that have been previously trained with training images in one or more training datasets.

14 . The method of claim 12 , wherein the inverse reshaping is performed with a reshaping mapping signaled in an image metadata portion for the forward reshaped image carried in the encoded video signal as a part of image metadata.

15 . An apparatus comprising a processor and configured to perform the method recited in claim 1 .

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (29)