Granted Patent B2

US 12,511,713 · App. 17/927,863 · Granted Dec 30, 2025

Inventors: Shuxin Zheng (Redmond, WA); Chang Liu (Redmond, WA); Di He (Redmond, WA); Guolin Ke (Redmond, WA); Jiang Bian (Redmond, WA); Tie-Yan Liu (Bejing, CN)

Assignee: Microsoft Technology Licensing, LLC

G06T3/4053G06T3/4046

Abstract

Claims (56)

1 . A computer-implemented method, comprising:

obtaining an input image of a first resolution;

training an invertible neural network with the input image, the invertible neural network being configured to generate an intermediate image of a second resolution and first high-frequency information based on the input image, the second resolution being lower than the first resolution;

wherein training an invertible neural network with the input image comprises:

determining a plurality of target functions based on the input image and the intermediate image by determining a first target function based on a difference between a pixel distribution in the intermediate image and a pixel distribution in an image block of the input image, the image block being of the second resolution; and

generating, using an inverse network of the trained invertible neural network, an output image of a third resolution based on second high-frequency information conforming to a predetermined distribution and the input image, the third resolution being greater than the first resolution.

2 . The method of claim 1 , wherein training an invertible neural network with the input image further comprises:

determining a total target function for training the invertible neural network by combining at least some of the plurality of target functions; and

determining network parameters for the invertible neural network by minimizing the total target function.

3 . The method of claim 2 , wherein determining the first target function comprises:

distinguishing, by using a discriminator, whether a pixel of the intermediate image belongs to the intermediate image or the image block; and

determining the first target function based on the distinguishing.

4 . The method of claim 2 , wherein determining a plurality of target functions comprises:

determining a second target function based on a difference between a distribution of the first high-frequency information and the predetermined distribution.

5 . The method of claim 2 , wherein determining a plurality of target functions comprises:

generating, by using an inverse network of the invertible neural network, a reconstructed image of the first resolution based on third high-frequency information conforming to the predetermined distribution and the intermediate image; and

determining a third target function based on a difference between the input image and the reconstructed image.

6 . The method of claim 2 , wherein determining the plurality of target functions comprises:

obtaining a reference image corresponding to semantics of the input image, the reference image being of the second resolution; and

determining a fourth target function based on a difference between the intermediate image and the reference image.

7 . The method of claim 1 , wherein the invertible neural network comprises a transforming module and at least one invertible network unit, and wherein generating the output image comprises:

generating, based on the input image and the second high-frequency information and by using the at least one invertible network unit, a low-frequency component and a high-frequency component to be merged, the low-frequency component representing semantics of the input image and the high-frequency component being related to the semantics; and

merging, by using the transforming module, the low-frequency component and the high-frequency component into the output image.

8 . The method of claim 7 , wherein the transforming module comprises at least one of:

an invertible convolution block; and

a wavelet transforming module.

9 . A device, comprising:

a processing unit; and

a memory coupled to the processing unit and comprising instructions stored thereon which, when executed by the processing unit, cause the device to perform acts comprising:

obtaining an input image of a first resolution;

training an invertible neural network with the input image, the invertible neural network being configured to generate an intermediate image of a second resolution and first high-frequency information based on the input image, the second resolution being lower than the first resolution;

wherein training an invertible neural network with the input image comprises:

determining a plurality of target functions based on the input image and the intermediate image by determining a first target function based on a difference between a pixel distribution in the intermediate image and a pixel distribution in an image block of the input image, the image block being of the second resolution; and

generating, using an inverse network of the trained invertible neural network, an output image of a third resolution based on second high-frequency information conforming to a predetermined distribution and the input image, the third resolution being greater than the first resolution.

10 . The device of claim 9 , wherein training an invertible neural network with the input image further comprises:

determining a total target function for training the invertible neural network by combining at least some of the plurality of target functions; and

determining network parameters for the invertible neural network by minimizing the total target function.

11 . The device of claim 10 , wherein determining the first target function comprises:

distinguishing, by using a discriminator, whether a pixel of the intermediate image belongs to the intermediate image or the image block; and

determining the first target function based on the distinguishing.

12 . The device of claim 10 , wherein determining a plurality of target functions comprises:

determining a second target function based on a difference between a distribution of the first high-frequency information and the predetermined distribution.

13 . A computer program product being tangibly stored in a non-transitory computer readable storage medium and comprising machine-executable instructions which, when executed by a device, cause the device to perform acts comprising:

obtaining an input image of a first resolution;

training an invertible neural network with the input image, the invertible neural network being configured to generate an intermediate image of a second resolution and first high-frequency information based on the input image, the second resolution being lower than the first resolution;

wherein training an invertible neural network with the input image comprises:

determining a plurality of target functions based on the input image and the intermediate image by determining a first target function based on a difference between a pixel distribution in the intermediate image and a pixel distribution in an image block of the input image, the image block being of the second resolution; and

generating, using an inverse network of the trained invertible neural network, an output image of a third resolution based on second high-frequency information conforming to a predetermined distribution and the input image, the third resolution being greater than the first resolution.

14 . The computer program product of claim 13 , wherein training an invertible neural network with the input image further comprises:

determining a total target function for training the invertible neural network by combining at least some of the plurality of target functions; and

determining network parameters for the invertible neural network by minimizing the total target function.

15 . The computer program product of claim 14 , wherein determining the first target function comprises:

distinguishing, by using a discriminator, whether a pixel of the intermediate image belongs to the intermediate image or the image block; and

determining the first target function based on the distinguishing.

16 . The computer program product of claim 14 , wherein determining a plurality of target functions comprises:

determining a second target function based on a difference between a distribution of the first high-frequency information and the predetermined distribution.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (19)