IP Library › Granted Patent US 12,430,725
Granted Patent B2
US 12,430,725 · App. 17/663,317 · Granted Sep 30, 2025

Object class inpainting in digital images utilizing class-specific inpainting neural networks

Inventors: Haitian Zheng (Rochester, NY); Zhe Lin (Fremont, CA); Jingwan Lu (Santa Clara, CA); Scott Cohen (Sunnyvale, CA); Elya Shechtman (Seattle, WA); Connelly Barnes (Seattle, WA); Jianming Zhang (Campbell, CA); Ning Xu (Milpitas, CA); Sohrab Amirghodsi (Seattle, WA)
Assignee: Adobe Inc.
G06T5/77G06N3/04G06T7/11G06T2207/20081G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,430,725
App. No.
17/663,317
Granted
Sep 30, 2025
Kind
B2
Abstract

The present disclosure relates to systems, methods, and non-transitory computer readable media that generate inpainted digital images utilizing class-specific cascaded modulation inpainting neural network. For example, the disclosed systems utilize a class-specific cascaded modulation inpainting neural network that includes cascaded modulation decoder layers to generate replacement pixels portraying a particular target object class. To illustrate, in response to user selection of a replacement region and target object class, the disclosed systems utilize a class-specific cascaded modulation inpainting neural network corresponding to the target object class to generate an inpainted digital image that portrays an instance of the target object class within the replacement region. Moreover, in one or more embodiments the disclosed systems train class-specific cascaded modulation inpainting neural networks corresponding to a variety of target object classes, such as a sky object class, a water object class, a ground object class, or a human object class.

Claims (52)

1. A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

receiving, via a user interface of a client device, an indication of a replacement region of a digital image and a target object class;

generating replacement pixels for the replacement region utilizing a class-specific inpainting neural network corresponding to the target object class and having a plurality of cascaded modulation layers, wherein a given cascaded modulation layer comprises a global modulation block and a spatial modulation block, by:

utilizing the global modulation blocks of the plurality of cascaded modulation layers to apply a modulation based on a global feature code to capture global predictions; and

utilizing the spatial modulation blocks of the plurality of cascaded modulation layers to apply a spatial modulation to refine the global predictions; and

providing, for display via the client device, an inpainted digital image comprising the replacement pixels such that the inpainted digital image portrays an instance of the target object class within the replacement region.

2. The non-transitory computer readable medium of claim 1 , wherein receiving the indication of the replacement region and the target object class comprises:

providing, for display via the user interface, the digital image; and

receiving, via the user interface, a user selection corresponding to the replacement region utilizing a selection tool corresponding to the target object class.

3. The non-transitory computer readable medium of claim 2 , further comprising: determining the replacement region utilizing a segmentation model and the user selection.

4. The non-transitory computer readable medium of claim 1 , further comprising instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

utilizing the global modulation block to generate a global feature map to apply a modulation based on a global feature code; and

utilizing the spatial modulation block to perform a spatial modulation utilizing the global feature map.

5. The non-transitory computer readable medium of claim 1 , further comprising instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising generating the replacement pixels utilizing the class-specific inpainting neural network corresponding to at least one of: a sky object class, a water object class, a ground object class, or a human object class.

6. The non-transitory computer readable medium of claim 1 , wherein generating the replacement pixels utilizing the class-specific inpainting neural network comprises generating an image encoding utilizing encoder layers of a class-specific cascaded modulation inpainting neural network.

7. The non-transitory computer readable medium of claim 6 , wherein generating the image encoding utilizing the encoder layers of the class-specific cascaded modulation inpainting neural network comprises:

generating positional encodings corresponding to different resolutions of the encoder layers; and

generating a plurality of encoding feature vectors utilizing the encoder layers and the positional encodings.

8. The non-transitory computer readable medium of claim 6 , wherein generating the replacement pixels comprises generating the replacement pixels utilizing cascaded modulation decoder layers of the class-specific cascaded modulation inpainting neural network from the image encoding.

9. A computer-implemented method comprising:

receiving, via a user interface of a client device, a user interaction with a digital image comprising an indication to replace a sky replacement region of the digital image;

determining, utilizing a panoptic segmentation model, a sky target object class based on the indication to replace the sky replacement region of a digital image;

selecting, based on the indicated sky target object class, a class-specific cascaded modulation inpainting neural network trained to generate sky regions for digital images;

generating sky replacement pixels for the sky replacement region utilizing the class-specific cascaded modulation inpainting neural network trained to generate sky regions for digital images and having a plurality of cascaded modulation layers, wherein a given cascaded modulation layer comprises a global modulation block and a spatial modulation block, by:

utilizing the global modulation blocks of the plurality of cascaded modulation layers to apply a modulation based on a global feature code to capture global predictions; and

utilizing the spatial modulation blocks of the plurality of cascaded modulation layers to apply a spatial modulation to refine the global predictions; and

providing, for display via the client device, an inpainted digital image comprising the sky replacement pixels within the sky replacement region.

10. The computer-implemented method of claim 9 , further comprising determining the sky replacement region based on user input selecting a portion of the digital image.

11. The computer-implemented method of claim 9 , further comprising:

selecting the class-specific cascaded modulation inpainting neural network trained to generate sky regions from a plurality of class-specific cascaded modulation inpainting neural networks based on the indication to replace the sky replacement region.

12. The computer-implemented method of claim 9 , further comprising generating the sky replacement pixels utilizing cascaded modulation decoder layers of the class-specific cascaded modulation inpainting neural network from an image encoding.

13. The computer-implemented method of claim 12 , wherein generating the sky replacement pixels comprises generating positional encodings corresponding to different resolutions of the cascaded modulation decoder layers.

14. The computer-implemented method of claim 13 , further comprising generating the sky replacement pixels utilizing the cascaded modulation decoder layers of the class-specific cascaded modulation inpainting neural network, the image encoding, and the positional encodings.

15. A system comprising:

one or more memory devices; and

one or more processors configured to cause the system to:

receive, via a user interface of a client device, an indication of a replacement region of a digital image and a target object class;

generate replacement pixels for the replacement region utilizing a class-specific inpainting neural network corresponding to the target object class and having a plurality of cascaded modulation layers, wherein a given cascaded modulation layer comprises a global modulation block and a spatial modulation block, by:

utilizing the global modulation blocks of the plurality of cascaded modulation layers to apply a modulation based on a global feature code to capture global predictions; and

utilizing the spatial modulation blocks of the plurality of cascaded modulation layers to apply a spatial modulation to refine the global predictions;

provide, for display via the client device, an inpainted digital image comprising the replacement pixels such that the inpainted digital image portrays an instance of the target object class within the replacement region.

16. The system of claim 15 , wherein receiving the indication of the replacement region and the target object class comprises:

providing, for display via the user interface, the digital image; and

receiving, via the user interface, a user selection corresponding to the replacement region utilizing a selection tool corresponding to the target object class.

17. The system of claim 16 , wherein the one or more processors are further configured to cause the system to determine the replacement region utilizing a segmentation model and the user selection.

18. The system of claim 15 , wherein the one or more processors are further configured to cause the system to:

utilizing the global modulation block to generate a global feature map to apply a modulation based on a global feature code; and

utilizing the spatial modulation block to perform a spatial modulation utilizing the global feature map.

19. The system of claim 15 , wherein generating the replacement pixels utilizing the class-specific inpainting neural network comprises generating an image encoding utilizing encoder layers of a class-specific cascaded modulation inpainting neural network.

20. The system of claim 19 , wherein generating the image encoding utilizing the encoder layers of the class-specific cascaded modulation inpainting neural network comprises:

generating positional encodings corresponding to different resolutions of the encoder layers; and

generating a plurality of encoding feature vectors utilizing the encoder layers and the positional encodings.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 13, 2022
From: ZHENG, HAITIAN; LIN, ZHE; LU, JINGWAN; COHEN, SCOTT; SHECHTMAN, ELYA; BARNES, CONNELLY; ZHANG, JIANMING; XU, NING; AMIRGHODSI, SOHRAB
To: ADOBE INC.
Reel/Frame 059902/0770 →
Continuity (1)
Related Publication 20230368339A1 · Nov 16, 2023
References Cited (93)
US 12056857B2 · Zhou · 2024 [cited by examiner]
US 20180374199A1 · Shen et al. · 2018 [cited by applicant]
US 20190196698A1 · Cohen et al. · 2019 [cited by applicant]
US 20200134834A1 · Pao et al. · 2020 [cited by applicant]
US 20200311874A1 · Ye et al. · 2020 [cited by applicant]
US 20210150682A1 · Sytnik et al. · 2021 [cited by applicant]
US 20210248721A1 · Tian et al. · 2021 [cited by applicant]
US 20210383242A1 · Ostyakov et al. · 2021 [cited by applicant]
US 20210390660A1 · Baek et al. · 2021 [cited by applicant]
US 20210407051A1 · Pardeshi et al. · 2021 [cited by applicant]
US 20220068037A1 · Pardeshi · 2022 [cited by examiner]
US 20220083806A1 · Cho · 2022 [cited by applicant]
US 20220180490A1 · Jo et al. · 2022 [cited by applicant]
US 20230196760A1 · Dorum et al. · 2023 [cited by applicant]
US 20230259587A1 · Lin · 2023 [cited by examiner]
US 20230351558A1 · Chen · 2023 [cited by examiner]
US 20230360180A1 · Zheng · 2023 [cited by examiner]
CN 109960453A · 2019 [cited by applicant]
CN 111354059A · 2020 [cited by applicant]
GB 2586678A · 2021 [cited by applicant]
KR 20200115001A · 2020 [cited by applicant]
WO 2021212810A1 · 2021 [cited by applicant]
Yu, Yingchen, et al. “Diverse image inpainting with bidirectional and autoregressive transformers.” Proceedings of the 29th ACM International Conference on Multimedia. 2021. [cited by examiner]
Roy, Hiya, et al. “Image inpainting using frequency-domain priors.” Journal of Electronic Imaging 30.2 (2021): 023016-023016. [cited by examiner]
Xu, Rui, et al. “Positional encoding as spatial inductive bias in gans.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. [cited by examiner]
Haoming Cai, Jingwen He, Yu Qiao and Chao Dong, “Toward Interactive Modulation for Photo-Realistic Image Restoration,” 2021 IEEE/CV Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Nashville, TN,… [cited by applicant]
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., and Aila, T., “Analyzing and Improving the Image Quality of StyleGAN”, arXiv e-prints, 2019. doi: 10.48550/arXiv.1912.04958 (Year: 2019). [cited by applicant]
Kim, H., Choi, Y., Kim, J., Yoo, S., and Uh, Y., “Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing”, arXiv e-prints, 2021. doi:10.48550/arXiv.2104.14754 (Year: 2021). [cited by applicant]
Suvorov, R., “Resolution-robust Large Mask Inpainting with Fourier Convolutions”, arXiv e-prints, 2021. doi: 10.48550/arXiv.2109.07161 (Year: 2021). [cited by applicant]
Xiao, Q., Li, G., and Chen, Q., “Deep Inception Generative Network for Cognitive Image Inpainting”, arXiv e-prints, 2018. doi:10.48550/arXiv.1812.01458 (Year: 2018). [cited by applicant]
Zhao, S., “Large Scale Image Completion via Co-Modulated Generative Adversarial Networks”, arXiv e-prints, 2021. doi: 10.48550/ arXiv.2103.10428 (Year: 2021). [cited by applicant]
Zhu, M., “Image Inpainting by End-to-End Cascaded Refinement With Mask Awareness”, IEEE Transactions on Image Processing, vol. 30, IEEE, pp. 4855-4866, 2021. doi:10.1109/TIP.2021.3076310 (Year: 2021). [cited by applicant]
U.S. Appl. No. 17/661,985, Jul. 26, 2024, Notice of Allowance. [cited by applicant]
U.S. Appl. No. 17/650,967, Sep. 6, 2024, Office Action. [cited by applicant]
Badour AlBahar, Jingwan Lu, Jimei Yang, Zhixin Shu, Eli Shechtman, and Jia-Bin Huang. Pose with Style: Detail-preserving pose-guided image synthesis with conditional stylegan. ACM Transactions on Graphics, 2021. [cited by applicant]
Jean-Francois Aujol, Guy Gilboa, Tony Chan, and Stanley Osher. Structure-texture image decomposition-modeling, algorithms, and parameter selection. International journal of computer vision, 67(1):111-136, 2006. [cited by applicant]
Coloma Ballester, Marcelo Bertalmio, Vicent Caselles, Guillermo Sapiro, and Joan Verdera. Filling-in by joint inter-polation of vector fields and gray levels. IEEE transactions on image processing, 10(8):1200-1211, 2001. [cited by applicant]
Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph., 28(3):24, 2009. [cited by applicant]
Marcelo Bertalmio, Luminita Vese, Guillermo Sapiro, and Stanley Osher. Simultaneous structure and texture image inpainting. IEEE transactions on image processing, 12(8):882-889, 2003. [cited by applicant]
Tony F Chan and Jianhong Shen. Nontexture inpainting by curvature-driven diffusions. Journal of visual communication and image representation, 12(4):436-449, 2001. [cited by applicant]
Lu Chi, Borui Jiang, and Yadong Mu. Fast fourier convolution. Advances in Neural Information Processing Systems, 33, 2020. [cited by applicant]
Taeg Sang Cho, Moshe Butman, Shai Avidan, and William T Freeman. The patch transform and its applications to image editing. In 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8. IEEE, 2008. [cited by applicant]
Antonio Criminisi, Patrick Perez, and Kentaro Toyama. Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on image processing, 13(9):1200-1212, 2004. [cited by applicant]
Alexei A Efros and William T Freeman. Image quilting for texture synthesis and transfer. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pp. 341-346. ACM, 2001. [cited by applicant]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pp. 267… [cited by applicant]
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of wasserstein gans. arXiv preprint arXiv:1704.00028, 2017. [cited by applicant]
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing … [cited by applicant]
Yibing Song Wei Huang Hongyu Liu, Bin Jiang and Chao Yang. Rethinking image inpainting via a mutual encoder-decoder with feature equalizations. InProceedings of the European Conference on Computer Vision, 2020. [cited by applicant]
Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. InProceedings of the IEEE International Conference on Computer Vision, pp. 1501-1510, 2017. [cited by applicant]
Xun Huang, Ming-Yu Liu, Serge Belongie, and Jan Kautz. Multimodal unsupervised image-to-image translation. In Proceedings of the European conference on computer vision (ECCV), pp. 172-189, 2018. [cited by applicant]
Satoshi lizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. Globally and locally consistent image completion. ACM Transactions on Graphics (ToG), 36(4):1-14, 2017. [cited by applicant]
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision, pp. 694-711. Springer, 2016. [cited by applicant]
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4401-4410, 20… [cited by applicant]
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of StyleGAN. InProc. CVPR, 2020. [cited by applicant]
Hyunsu Kim, Yunjey Choi, Junho Kim, Sungjoo Yoo, and Youngjung Uh. Exploiting spatial dimensions of latent in gan for real-time image editing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni… [cited by applicant]
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014. [cited by applicant]
Vivek Kwatra, Irfan Essa, Aaron Bobick, and Nipun Kwatra. Texture optimization for example-based synthesis. In ACM SIGGRAPH 2005 Papers, pp. 795-802. 2005. [cited by applicant]
Yanwei Li, Hengshuang Zhao, Xiaojuan Qi, Liwei Wang, Zeming Li, Jian Sun, and Jiaya Jia. Fully convolutional networks for panoptic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R… [cited by applicant]
Guilin Liu, Fitsum A Reda, Kevin J Shih, Ting-Chun Wang, Andrew Tao, and Bryan Catanzaro. Image inpainting for irregular holes using partial convolutions. In Proceedings of the European Conference on Computer Vision (EC… [cited by applicant]
Wenjie Luo, Yujia Li, Raquel Urtasun, and Richard Zemel. Understanding the effective receptive field in deep convolutional neural networks. In Proceedings of the 30th International Conference on Neural Information Proce… [cited by applicant]
Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for gans do actually converge? In International conference on machine learning, pp. 3481-3490. PMLR, 2018. [cited by applicant]
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957, 2018. [cited by applicant]
Kamyar Nazeri, Eric Ng, Tony Joseph, Faisal Z Qureshi, and Mehran Ebrahimi. Edgeconnect: Generative image inpainting with adversarial edge learning. arXiv preprint arXiv:1901.00212, 2019. [cited by applicant]
Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu. Semantic image synthesis with spatially-adaptive normalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019. [cited by applicant]
Taesung Park, Jun-Yan Zhu, Oliver Wang, Jingwan Lu, Eli Shechtman, Alexei A Efros, and Richard Zhang. Swapping autoencoder for deep image manipulation. arXiv preprint arXiv:2007.00653, 2020. [cited by applicant]
Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp… [cited by applicant]
Jialun Peng, Dong Liu, Songcen Xu, and Houqiang Li. Generating diverse structure for image inpainting with hierarchical vq-vae. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)… [cited by applicant]
Yurui Ren, Xiaoming Yu, Ruonan Zhang, Thomas H. Li, Shan Liu, and Ge Li. Structureflow: Image inpainting via structure-aware appearance flow. In IEEE International Conference on Computer Vision (ICCV), 2019. [cited by applicant]
Tim Salimans and Durk P Kingma. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. Advances in neural information processing systems, 29:901-909, 2016. [cited by applicant]
Jianhong Shen and Tony F Chan. Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics, 62(3):1019-1043, 2002. [cited by applicant]
Yuhang Song, Chao Yang, Yeji Shen, Peng Wang, Qin Huang, and C-C Jay Kuo. Spg-net: Segmentation prediction and guidance network for image inpainting.arXiv preprint arXiv:1805.03356, 2018. [cited by applicant]
Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with … [cited by applicant]
Zhentao Tan, Dongdong Chen, Qi Chu, Menglei Chai, Jing Liao, Mingming He, Lu Yuan, Gang Hua, and Nenghai Yu. Semantic image synthesis via efficient class-adaptive normalization.arXiv preprint arXiv:2012.04644, 2020. [cited by applicant]
Ziyu Wan, Jingbo Zhang, Dongdong Chen, and Jing Liao. High-fidelity pluralistic image completion with transformers.arXiv preprint arXiv:2103.14031, 2021. [cited by applicant]
Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. Recovering realistic texture in image super-resolution by deep spatial feature transform. In Proceedings of the IEEE conference on computer vision and pattern recognit… [cited by applicant]
Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, et al. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600-612, 2004. [cited by applicant]
Wei Xiong, Jiahui Yu, Zhe Lin, Jimei Yang, Xin Lu, Connelly Barnes, and Jiebo Luo. Foreground-aware image inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5840-5848, … [cited by applicant]
Chao Yang, Xin Lu, Zhe Lin, Eli Shechtman, Oliver Wang, and Hao Li. High-resolution image inpainting using multi-scale neural patch synthesis. In2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017… [cited by applicant]
Jie Yang, Zhiquan Qi, and Yong Shi. Learning to incorporate structure knowledge for image inpainting. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12605-12612, 2020. [cited by applicant]
Zili Yi, Qiang Tang, Shekoofeh Azizi, Daesik Jang, and Zhan Xu. Contextual residual aggregation for ultra high-resolution image inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn… [cited by applicant]
Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Generative image inpainting with contextual attention. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5505… [cited by applicant]
Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Free-form image inpainting with gated convolution. InProceedings of the IEEE International Conference on Computer Vision, pp. 4471-4480, 2019. [cited by applicant]
Yu Zeng, Zhe Lin, Huchuan Lu, and Vishal M. Patel. Cr-fill: Generative image inpainting with auxiliary contextual reconstruction. InProceedings of the IEEE International Conference on Computer Vision, 2021. [cited by applicant]
Yu Zeng, Zhe Lin, Jimei Yang, Jianming Zhang, Eli Shechtman, and Huchuan Lu. High-resolution image inpainting with iterative confidence feedback and guided upsampling. arXiv preprint arXiv:2005.11742, 2020. [cited by applicant]
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern … [cited by applicant]
Shengyu Zhao, Jonathan Cui, Yilun Sheng, Yue Dong, Xiao Liang, Eric I Chang, and Yan Xu. Large scale image completion via co-modulated generative adversarial networks. arXiv preprint arXiv:2103.10428, 2021. [cited by applicant]
Haitian Zheng, Haofu Liao, Lele Chen, Wei Xiong, Tianlang Chen, and Jiebo Luo. Example-guided image synthesis using masked spatial-channel attention and self-supervision. In European Conference on Computer Vision, pp. 4… [cited by applicant]
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452-1464, 2… [cited by applicant]
Github; PanopticFCN: Fully Convolutional Networks for Panoptic Segmentation; Date downloaded May 26, 2022; https://github.com/dvlab-research/PanopticFCN. [cited by applicant]
Search and Examination Report as received in GB application 2303646.0 dated Oct. 3, 2023. [cited by applicant]
U.S. Appl. No. 17/650,967, Nov. 27, 2024, Notice of Allowance. [cited by applicant]
Office Action as received in Chinese Application No. 202310157677.5 dated Jul. 26, 2025. [cited by applicant]
Yin Wei-wei, Li Rong-wei, Wu Le-nan. “Joint Source-Channel Coding/Modulation Based on NN”, Journal of Agglied Sciences, vol. 23, No. 1, 3 pages, Jan. 2005. [cited by applicant]
Cited By (1)
US 12,647,611