IP Library Granted Patent US 12,254,064
Granted Patent B2
US 12,254,064 · App. 17/488,735 · Granted Mar 18, 2025

Image generation method, neural network compression method, and related apparatus and device

Inventors: Hanting Chen (Beijing, CN); Yunhe Wang (Beijing, CN); Chuanjian Liu (Beijing, CN); Kai Han (Beijing, CN); Chunjing Xu (Shenzhen, CN)
Assignee: HUAWEI TECHNOLOGIES CO., LTD.
G06F18/2148G06F18/243G06N3/047
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,254,064
App. No.
17/488,735
Granted
Mar 18, 2025
Kind
B2
Abstract

The present application discloses an image generation method, a neural network compression method, and a related apparatus and device in the field of artificial intelligence. The image generation method includes: inputting a first matrix into an initial image generator to obtain a generated image; inputting the generated image into a preset discriminator to obtain a determining result, where the preset discriminator is obtained through training based on a real image and a category corresponding to the real image; updating the initial image generator based on the determining result to obtain a target image generator; and further inputting a second matrix into the target image generator to obtain a sample image. Further, a neural network compression method is disclosed, to compress the preset discriminator based on the sample image obtained by using the foregoing image generation method.

Claims (65)

1. An image generation method, comprising:

inputting a first matrix into an initial image generator to obtain a generated image, wherein the initial image generator is a deep neural network, and the first matrix does not contain image data;

inputting the generated image into a preset discriminator to obtain a determining result, wherein the preset discriminator is obtained through training based on first training data, and the first training data comprises a real image and a category corresponding to the real image;

updating the initial image generator based on the determining result to obtain a target image generator; and

inputting a second matrix into the target image generator to obtain a sample image, wherein the second matrix is a same type and format as the first matrix.

2. The method according to claim 1 , wherein the determining result comprises a probability that the generated image is predicted as each of M categories, and M is an integer greater than 1.

3. The method according to claim 2 , wherein the updating the initial image generator based on the determining result specifically comprises:

determining a maximum probability in probabilities corresponding to the M categories, and determining, as a real result of the generated image, a category corresponding to the maximum probability; and

updating the initial image generator based on a difference between the determining result and the real result.

4. The method according to claim 2 , wherein before the updating the initial image generator based on the determining result, the method further comprises:

extracting a feature of the generated image by using the preset discriminator; and

the updating the initial image generator based on the determining result specifically comprises:

determining a maximum probability in probabilities corresponding to the M categories, and determining, as a real result of the generated image, a category corresponding to the maximum probability; and updating the initial image generator based on the feature and a difference between the determining result and the real result.

5. The method according to claim 2 , wherein the updating the initial image generator based on the determining result specifically comprises:

determining a maximum probability in probabilities corresponding to the M categories, and determining, as a real result of the generated image, a category corresponding to the maximum probability;

obtaining, based on N determining results that one-to-one correspond to N first matrices, an average probability value of each of the M categories in the N determining results, wherein N is a positive integer; and

updating the initial image generator based on the average probability value and a difference between the determining result and the real result.

6. The method according to claim 2 , wherein before the updating the initial image generator based on the determining result, the method further comprises:

extracting a feature of the generated image by using the preset discriminator; and

the updating the initial image generator based on the determining result specifically comprises:

determining a maximum probability in probabilities corresponding to the M categories, and determining, as a real result of the generated image, a category corresponding to the maximum probability; obtaining, based on N determining results that one-to-one correspond to N first matrices, an average probability value of each of the M categories in the N determining results; and updating the initial image generator based on a difference between the determining result and the real result, an eigenvalue of the feature, and the average probability value.

7. A sample generation method, comprising:

inputting a first matrix into an initial sample generator to obtain a first generated sample, wherein the initial sample generator is a deep neural network, and the first matrix does not contain image data;

inputting the first generated sample into a preset discriminator to obtain a determining result, wherein the preset discriminator is obtained through training based on first training data, and the first training data comprises a real sample and a category corresponding to the real sample;

updating parameters of the initial sample generator based on the determining result of the first generated sample to obtain a target sample generator; and

inputting a second matrix into the target sample generator to obtain a second generated sample, wherein the second matrix is a same type and format as the first matrix.

8. The method according to claim 7 , wherein the determining result comprises a probability that the first generated sample is predicted as each of M categories, and M is an integer greater than 1.

9. The method according to claim 8 , wherein the updating the initial sample generator based on the determining result specifically comprises:

determining a maximum probability in probabilities corresponding to the M categories, and determining, as a real result of the first generated sample, a category corresponding to the maximum probability; and

updating the initial sample generator based on a difference between the determining result and the real result.

10. The method according to claim 8 , wherein before the updating the initial sample generator based on the determining result, the method further comprises:

extracting a feature of the first generated sample by using the preset discriminator; and

the updating the initial sample generator based on the determining result specifically comprises:

determining a maximum probability in probabilities corresponding to the M categories, and determining, as a real result of the first generated sample, a category corresponding to the maximum probability; and updating the initial sample generator based on the feature and a difference between the determining result and the real result.

11. The method according to claim 8 , wherein the updating the initial sample generator based on the determining result specifically comprises:

determining a maximum probability in probabilities corresponding to the M categories, and determining, as a real result of the first generated sample, a category corresponding to the maximum probability;

obtaining, based on N determining results that one-to-one correspond to N first matrices, an average probability value of each of the M categories in the N determining results, wherein N is a positive integer; and

updating the initial sample generator based on the average probability value and a difference between the determining result and the real result.

12. The method according to claim 8 , wherein before the updating the initial sample generator based on the determining result, the method further comprises:

extracting a feature of the first generated sample by using the preset discriminator; and

the updating the initial sample generator based on the determining result specifically comprises:

determining a maximum probability in probabilities corresponding to the M categories, and determining, as a real result of the first generated sample, a category corresponding to the maximum probability; obtaining, based on N determining results that one-to-one correspond to N first matrices, an average probability value of each of the M categories in the N determining results; and updating the initial sample generator based on a difference between the determining result and the real result, the feature, and the average probability value.

13. An image generation apparatus, comprising a memory and a processor, wherein the memory is configured to store a program, the processor is configured to execute the program stored in the memory, and when the program stored in the memory is executed, the processor is configured to perform an image generation method, the method comprising:

inputting a first matrix into an initial image generator to obtain a generated image, wherein the initial image generator is a deep neural network, and the first matrix does not contain image data;

inputting the generated image into a preset discriminator to obtain a determining result, wherein the preset discriminator is obtained through training based on first training data, and the first training data comprises a real image and a category corresponding to the real image;

updating the initial image generator based on the determining result to obtain a target image generator; and

inputting a second matrix into the target image generator to obtain a sample image, wherein the second matrix is a same type and format as the first matrix.

14. The apparatus according to claim 13 , wherein the determining result comprises a probability that the generated image is predicted as each of M categories, and M is an integer greater than 1.

15. The apparatus according to claim 14 , wherein the updating the initial image generator based on the determining result specifically comprises:

determining a maximum probability in probabilities corresponding to the M categories, and determining, as a real result of the generated image, a category corresponding to the maximum probability; and

updating the initial image generator based on a difference between the determining result and the real result.

16. The apparatus according to claim 14 , wherein before the updating the initial image generator based on the determining result, the method further comprises:

extracting a feature of the generated image by using the preset discriminator; and

the updating the initial image generator based on the determining result specifically comprises:

determining a maximum probability in probabilities corresponding to the M categories, and determining, as a real result of the generated image, a category corresponding to the maximum probability; and updating the initial image generator based on the feature and a difference between the determining result and the real result.

17. The apparatus according to claim 14 , wherein the updating the initial image generator based on the determining result specifically comprises:

determining a maximum probability in probabilities corresponding to the M categories, and determining, as a real result of the generated image, a category corresponding to the maximum probability;

obtaining, based on N determining results that one-to-one correspond to N first matrices, an average probability value of each of the M categories in the N determining results, wherein N is a positive integer; and

updating the initial image generator based on the average probability value and a difference between the determining result and the real result.

18. The apparatus according to claim 14 , wherein before the updating the initial image generator based on the determining result, the method further comprises:

extracting a feature of the generated image by using the preset discriminator; and

the updating the initial image generator based on the determining result specifically comprises:

determining a maximum probability in probabilities corresponding to the M categories, and determining, as a real result of the generated image, a category corresponding to the maximum probability; obtaining, based on N determining results that one-to-one correspond to N first matrices, an average probability value of each of the M categories in the N determining results; and updating the initial image generator based on a difference between the determining result and the real result, an eigenvalue of the feature, and the average probability value.

19. A computer-readable storage medium, wherein the computer-readable storage medium stores program code to be executed by a device, and the program code comprises the image generation method according to claim 1 .

20. A computer-readable storage medium, wherein the computer-readable storage medium stores program code to be executed by a device, and the program code comprises the sample generation method according to claim 7 .

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 2, 2021
From: CHEN, HANTING; WANG, YUNHE; LIU, CHUANJIAN; HAN, KAI; XU, CHUNJING
To: HUAWEI TECHNOLOGIES CO., LTD.
Reel/Frame 058268/0421 →
Priority Claims (1)
CN 201910254752.3 · Mar 31, 2019 · national
Continuity (2)
Continuation PCTCN2020082599 · Mar 31, 2020
Related Publication 20220019855A1 · Jan 20, 2022
References Cited (20)
US 20180336439A1 · Kliger · 2018 [cited by examiner]
US 20180336471A1 · Rezagholizadeh · 2018 [cited by examiner]
US 20200202174A1 · Lin · 2020 [cited by examiner]
US 20210374911A1 · Kudo · 2021 [cited by examiner]
CN 107563428A · 2018 [cited by applicant]
CN 109284786A · 2019 [cited by applicant]
CN 109344921A · 2019 [cited by applicant]
CN 109508669A · 2019 [cited by applicant]
CN 109543753A · 2019 [cited by applicant]
CN 110084281A · 2019 [cited by applicant]
Yaxing Wang et al.,“Transferring GANs: generating images from limited data”, Oct. 2, 2018,total:17pages. [cited by applicant]
Hui Jonathan et al., “CAN what is generative Adversarial Networks can”,Jun. 19, 2018,total:9pages. [cited by applicant]
Tim Salimans et al., “Improved Techniques for Training GANs”,Jun. 10, 2016,total:10pages. [cited by applicant]
Yang Zhao et al., “Self-Adversarially Learned Bayesian Sampling”,Nov. 21, 2018,total:8pages. [cited by applicant]
Vasileios Belagiannis et al., “Adversarial Network Compression”,2019year, total:19pages. [cited by applicant]
Huyeeeee et al., “Transferring GANs: generating images from limited data The understanding of the paper”,Nov. 26, 2018,total:5pages. [cited by applicant]
Anonymous CVPR submission, No Free-Lunch Privacy in Shared Deep Neural Networks, CVPR 2019 Submission #965, 2019, 9 pages. [cited by applicant]
Augustus Odena: “Semi-Supervised Learning with Generative Adversarial Networks”, arxiv.org, Cornell University Library, 201 Olin Library Cornell University Ithaca, NY 14853, Jun. 6, 2016 (Jun. 6, 2016), XP080706029, 3 p… [cited by applicant]
Raphael Gontijo Lopes et al: “Data-Free Knowledge Distillation for Deep Neural Networks”, arxiv.org, Cornell University Library, 201 Olin Library Cornell University Ithaca, NY 14853, Oct. 19, 2017 (Oct. 19, 2017), XP081… [cited by applicant]
Cheng Yu et al: “IEEE Signal Processing Magazine, Special Issue on Deep Learning for Image Understanding (Arxiv Extended Version) A Survey of Model Compression and Acceleration for Deep Neural Networks”, Feb. 7, 2019 (F… [cited by applicant]
Cited By (1)
US 12,315,229