IP Library › Granted Patent US 12,204,610
Granted Patent B2
US 12,204,610 · App. 17/650,967 · Granted Jan 21, 2025

Learning parameters for generative inpainting neural networks utilizing object-aware training and masked regularization

Inventors: Zhe Lin (Fremont, CA); Haitian Zheng (Rochester, NY); Jingwan Lu (Santa Clara, CA); Scott Cohen (Sunnyvale, CA); Jianming Zhang (Campbell, CA); Ning Xu (Milpitas, CA); Elya Shechtman (Seattle, WA); Connelly Barnes (Seattle, WA); Sohrab Amirghodsi (Seattle, WA)
Assignee: Adobe Inc.
G06F18/2148G06N3/08G06T5/77G06T7/11G06T2207/20081G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,204,610
App. No.
17/650,967
Granted
Jan 21, 2025
Kind
B2
Abstract

The present disclosure relates to systems, methods, and non-transitory computer readable media for training a generative inpainting neural network to accurately generate inpainted digital images via object-aware training and/or masked regularization. For example, the disclosed systems utilize an object-aware training technique to learn parameters for a generative inpainting neural network based on masking individual object instances depicted within sample digital images of a training dataset. In some embodiments, the disclosed systems also (or alternatively) utilize a masked regularization technique as part of training to prevent overfitting by penalizing a discriminator neural network utilizing a regularization term that is based on an object mask. In certain cases, the disclosed systems further generate an inpainted digital image utilizing a trained generative inpainting model with parameters learned via the object-aware training and/or the masked regularization.

Claims (52)

1. A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor, cause a computing device to perform operations comprising:

generating a set of object masks for objects within a digital image utilizing a segmentation model;

selecting a masked digital image from a set of masked digital images depicting masked object instances indicated by the set of object masks for the digital image;

generating an inpainted digital image from the masked digital image by filling a hole region indicated by a digital image mask of the masked digital image utilizing a generative inpainting neural network comprising a discriminator neural network;

comparing the inpainted digital image with the digital image utilizing a masked regularization from the digital image mask to penalize the discriminator neural network from overfitting by enforcing computation of gradient penalties on unmasked pixels outside of the digital image mask; and

modifying parameters of the generative inpainting neural network based on the inpainted digital image according to the masked regularization.

2. The non-transitory computer readable medium of claim 1 , further storing instructions that, when executed by the at least one processor, cause the computing device to perform operations comprising:

determining an overlap ratio between a digital image mask of the masked digital image and a masked object instance of the masked object instances;

comparing the overlap ratio with an overlap ratio threshold; and

modifying the digital image mask to exclude the masked object instance based on comparing the overlap ratio with the overlap ratio threshold.

3. The non-transitory computer readable medium of claim 1 , wherein generating the set of object masks utilizing the segmentation model comprises generating object masks corresponding to individual object instances depicted within the digital image utilizing a panoptic segmentation neural network.

4. The non-transitory computer readable medium of claim 1 , further storing instructions that, when executed by the at least one processor, cause the computing device to generate the set of masked digital images to include masked digital images depicting masked object instances and additional masked digital images depicting random pattern masks.

5. The non-transitory computer readable medium of claim 1 , further storing instructions that, when executed by the at least one processor, cause the computing device to perform operations comprising:

determining a perceptual loss; and

modifying the parameters of the generative inpainting neural network according to the perceptual loss.

6. The non-transitory computer readable medium of claim 2 , further storing instructions that, when executed by the at least one processor, cause the computing device to perform operations comprising reducing overfitting by the generative inpainting neural network by dilating and translating the masked object instance.

7. The non-transitory computer readable medium of claim 1 , further storing instructions that, when executed by the at least one processor, cause the computing device to perform operations comprising reducing leaking of background pixels into a hole region of the digital image indicated by the digital image mask by dilating the digital image mask along a segmentation boundary indicated by the set of object masks.

8. A system comprising:

one or more memory devices comprising a digital image and a generative inpainting neural network comprising a discriminator neural network; and

one or more processors configured to cause the system to:

generate a digital image mask indicating a hole region within the digital image;

generate an inpainted digital image from the digital image by inpainting the hole region utilizing the generative inpainting neural network;

compare the inpainted digital image with the digital image utilizing a masked regularization from the digital image mask to penalize the discriminator neural network from overfitting by enforcing computation of gradient penalties on unmasked pixels outside of the digital image mask; and

modify parameters of the generative inpainting neural network based on comparing the inpainted digital image with the digital image according to the masked regularization.

9. The system of claim 8 , wherein utilizing the masked regularization to penalize the discriminator neural network comprises utilizing an R1 regularization that incorporates the digital image mask.

10. The system of claim 8 , wherein the one or more processors are further configured to cause the system to:

generate a set of object masks indicating objects depicted within the digital image; and

generate the digital image mask by generating a masked object instance corresponding to an object instance from among the objects depicted within the digital image.

11. The system of claim 10 , wherein the one or more processors are further configured to cause the system to:

determine an overlap ratio between the digital image mask and the masked object instance;

generate a modified digital image mask from the digital image mask according to the overlap ratio; and

generate the inpainted digital image by inpainting a modified hole region indicated by the modified digital image mask.

12. The system of claim 11 , wherein the one or more processors are further configured to cause the system to determine the overlap ratio by comparing mask pixels occupied by the digital image mask with segmentation pixels occupied by the masked object instance.

13. The system of claim 8 , wherein the one or more processors are further configured to cause the system to generate the digital image mask by generating one or more of a random pattern mask comprising a combination of random strokes and shapes or a masked object instance shaped like an object mask of an object instance depicted within the digital image.

14. The system of claim 8 , wherein comparing the inpainted digital image with a digital image comprises comparing the inpainted digital image with an unmodified version of the digital image without the hole region.

15. A computer-implemented method comprising:

generating a set of object masks for objects within a digital image utilizing a segmentation model;

selecting a masked digital image from a set of masked digital images depicting masked object instances indicated by the set of object masks for the digital image;

generating an inpainted digital image from the masked digital image by filling a hole region indicated by a digital image mask of the masked digital image utilizing a generative inpainting neural network comprising a discriminator neural network;

comparing the inpainted digital image with the digital image utilizing a masked regularization from the digital image mask to penalize the discriminator neural network from overfitting by enforcing computation of gradient penalties on unmasked pixels outside of the digital image mask; and

modifying parameters of the generative inpainting neural network based on the inpainted digital image according to the masked regularization.

16. The computer-implemented method of claim 15 , further comprising:

determining an overlap ratio between a digital image mask of the masked digital image and a masked object instance of the masked object instances;

comparing the overlap ratio with an overlap ratio threshold; and

modifying the digital image mask to exclude the masked object instance based on comparing the overlap ratio with the overlap ratio threshold.

17. The computer-implemented method of claim 15 , wherein generating the set of object masks utilizing the segmentation model comprises generating object masks corresponding to individual object instances depicted within the digital image utilizing a panoptic segmentation neural network.

18. The computer-implemented method of claim 15 , further comprising:

generating the set of masked digital images to include masked digital images depicting masked object instances and additional masked digital images depicting random pattern masks.

19. The computer-implemented method of claim 15 , further comprising:

determining a perceptual loss; and

modifying the parameters of the generative inpainting neural network according to the perceptual loss.

20. The computer-implemented method of claim 16 , further comprising reducing overfitting by the generative inpainting neural network by dilating and translating the masked object instance.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 14, 2022
From: LIN, ZHE; ZHENG, HAITIAN; LU, JINGWAN; COHEN, SCOTT; ZHANG, JIANMING; XU, NING; SHECHTMAN, ELYA; BARNES, CONNELLY; AMIRGHODSI, SOHRAB
To: ADOBE INC.
Reel/Frame 059005/0315 →
Continuity (1)
Related Publication 20230259587A1 · Aug 17, 2023
References Cited (88)
US 12056857B2 · Zhou et al. · 2024 [cited by applicant]
US 20180374199A1 · Shen et al. · 2018 [cited by applicant]
US 20190196698A1 · Cohen · 2019 [cited by examiner]
US 20200134834A1 · Pao et al. · 2020 [cited by applicant]
US 20210150682A1 · Sytnik · 2021 [cited by examiner]
US 20210248721A1 · Tian et al. · 2021 [cited by applicant]
US 20210383242A1 · Ostyakov · 2021 [cited by examiner]
US 20210390660A1 · Baek et al. · 2021 [cited by applicant]
US 20210407051A1 · Pardeshi et al. · 2021 [cited by applicant]
US 20220068037A1 · Pardeshi et al. · 2022 [cited by applicant]
US 20220083806A1 · Cho · 2022 [cited by examiner]
US 20220180490A1 · Jo · 2022 [cited by examiner]
US 20230196760A1 · Dorum · 2023 [cited by examiner]
US 20230259587A1 · Lin et al. · 2023 [cited by applicant]
US 20230360180A1 · Zheng et al. · 2023 [cited by applicant]
CN 111354059A · 2020 [cited by applicant]
GB 2586678A · 2021 [cited by examiner]
WO 2021212810A1 · 2021 [cited by applicant]
Yu, Jiahui, et al. “Generative image inpainting with contextual attention.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. (Year: 2018). [cited by examiner]
Xiong, Wei, et al. “Foreground-aware image inpainting.” Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019. (Year: 2019). [cited by examiner]
Suvorov, Roman, et al. “Resolution-robust Large Mask Inpainting with Fourier Convolutions.” arXiv preprint arXiv: 2109.07161. (Year: 2021). [cited by examiner]
Github; PanopticFCN: Fully Convolutional Networks for Panoptic Segmentation; Date downloaded May 26, 2022; https://github.com/dvlab-research/PanopticFCN. [cited by applicant]
Search and Examination Report as received in GB application 2303646.0 dated Oct. 3, 2023. [cited by applicant]
Badour AlBahar, Jingwan Lu, Jimei Yang, Zhixin Shu, Eli Shechtman, and Jia-Bin Huang. Pose with Style: Detail-preserving pose-guided image synthesis with conditional stylegan. ACM Transactions on Graphics, 2021. [cited by applicant]
Jean-Francois Aujol, Guy Gilboa, Tony Chan, and Stanley Osher. Structure-texture image decomposition—modeling, algorithms, and parameter selection. International journal of computer vision, 67(1):111-136, 2006. [cited by applicant]
Coloma Ballester, Marcelo Bertalmio, Vicent Caselles, Guillermo Sapiro, and Joan Verdera. Filling-in by joint interpolation of vector fields and gray levels. IEEE transactions on image processing, 10(8):1200-1211, 2001. [cited by applicant]
Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph., 28(3):24, 2009. [cited by applicant]
Marcelo Bertalmio, Luminita Vese, Guillermo Sapiro, and Stanley Osher. Simultaneous structure and texture image inpainting. IEEE transactions on image processing, 12(8):882-889, 2003. [cited by applicant]
Tony F Chan and Jianhong Shen. Nontexture inpainting by curvature-driven diffusions. Journal of visual communication and image representation, 12(4):436-449, 2001. [cited by applicant]
Lu Chi, Borui Jiang, and Yadong Mu. Fast fourier convolution. Advances in Neural Information Processing Systems, 33, 2020. [cited by applicant]
Taeg Sang Cho, Moshe Butman, Shai Avidan, and William T Freeman. The patch transform and its applications to image editing. In 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8. IEEE, 2008. [cited by applicant]
Antonio Criminisi, Patrick Perez, and Kentaro Toyama. Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on image processing, 13(9):1200-1212, 2004. [cited by applicant]
Alexei A Efros and William T Freeman. Image quilting for texture synthesis and transfer. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pp. 341-346. ACM, 2001. [cited by applicant]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pp. 267… [cited by applicant]
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of wasserstein gans. arXiv preprint arXiv:1704.00028, 2017. [cited by applicant]
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing sys… [cited by applicant]
Yibing Song Wei Huang Hongyu Liu, Bin Jiang and Chao Yang. Rethinking image inpainting via a mutual encoder-decoder with feature equalizations. In Proceedings of the European Conference on Computer Vision, 2020. [cited by applicant]
Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1501-1510, 2017. [cited by applicant]
Xun Huang, Ming-Yu Liu, Serge Belongie, and Jan Kautz. Multimodal unsupervised image-to-image translation. In Proceedings of the European conference on computer vision (ECCV), pp. 172-189, 2018. [cited by applicant]
Satoshi Iizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. Globally and locally consistent image completion. ACM Transactions on Graphics (ToG), 36(4):1-14, 2017. [cited by applicant]
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision, pp. 694-711. Springer, 2016. [cited by applicant]
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4401-4410, 20… [cited by applicant]
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of StyleGAN. In Proc. CVPR, 2020. [cited by applicant]
Hyunsu Kim, Yunjey Choi, Junho Kim, Sungjoo Yoo, and Youngjung Uh. Exploiting spatial dimensions of latent in gan for real-time image editing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni… [cited by applicant]
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014. [cited by applicant]
Vivek Kwatra, Irfan Essa, Aaron Bobick, and Nipun Kwatra. Texture optimization for example-based synthesis. In ACM SIGGRAPH 2005 Papers, pp. 795-802. 2005. [cited by applicant]
Yanwei Li, Hengshuang Zhao, Xiaojuan Qi, Liwei Wang, Zeming Li, Jian Sun, and Jiaya Jia. Fully convolutional networks for panoptic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R… [cited by applicant]
Guilin Liu, Fitsum A Reda, Kevin J Shih, Ting-Chun Wang, Andrew Tao, and Bryan Catanzaro. Image inpainting for irregular holes using partial convolutions. In Proceedings of the European Conference on Computer Vision (EC… [cited by applicant]
Wenjie Luo, Yujia Li, Raquel Urtasun, and Richard Zemel. Understanding the effective receptive field in deep convolutional neural networks. InProceedings of the 30th International Conference on Neural Information Proces… [cited by applicant]
Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for gans do actually converge? In International conference on machine learning, pp. 3481-3490. PMLR, 2018. [cited by applicant]
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks.arXiv preprint arXiv:1802.05957, 2018. [cited by applicant]
Kamyar Nazeri, Eric Ng, Tony Joseph, Faisal Z Qureshi, and Mehran Ebrahimi. Edgeconnect: Generative image inpainting with adversarial edge learning.arXiv preprint arXiv:1901.00212, 2019. [cited by applicant]
Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu. Semantic image synthesis with spatially-adaptive normalization. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019. [cited by applicant]
Taesung Park, Jun-Yan Zhu, Oliver Wang, Jingwan Lu, Eli Shechtman, Alexei A Efros, and Richard Zhang. Swapping autoencoder for deep image manipulation.arXiv preprint arXiv:2007.00653, 2020. [cited by applicant]
Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp… [cited by applicant]
Jialun Peng, Dong Liu, Songcen Xu, and Houqiang Li. Generating diverse structure for image inpainting with hierarchical vq-vae. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)… [cited by applicant]
Yurui Ren, Xiaoming Yu, Ruonan Zhang, Thomas H. Li, Shan Liu, and Ge Li. Structureflow: Image inpainting via structure-aware appearance flow. In IEEE International Conference on Computer Vision (ICCV), 2019. [cited by applicant]
Tim Salimans and Durk P Kingma. Weight normalization: A simple reparameterization to accelerate training of deep neural networks.Advances in neural information processing systems, 29:901-909, 2016. [cited by applicant]
Jianhong Shen and Tony F Chan. Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics, 62(3):1019-1043, 2002. [cited by applicant]
Yuhang Song, Chao Yang, Yeji Shen, Peng Wang, Qin Huang, and C-C Jay Kuo. Spg-net: Segmentation prediction and guidance network for image inpainting. arXiv preprint arXiv:1805.03356, 2018. [cited by applicant]
Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with … [cited by applicant]
Zhentao Tan, Dongdong Chen, Qi Chu, Menglei Chai, Jing Liao, Mingming He, Lu Yuan, Gang Hua, and Nenghai Yu. Semantic image synthesis via efficient class-adaptive normalization.arXiv preprint arXiv:2012.04644, 2020. [cited by applicant]
Ziyu Wan, Jingbo Zhang, Dongdong Chen, and Jing Liao. High-fidelity pluralistic image completion with transformers. arXiv preprint arXiv:2103.14031, 2021. [cited by applicant]
Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. Recovering realistic texture in image super-resolution by deep spatial feature transform. In Proceedings of the IEEE conference on computer vision and pattern recognit… [cited by applicant]
Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, et al. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600-612, 2004. [cited by applicant]
Wei Xiong, Jiahui Yu, Zhe Lin, Jimei Yang, Xin Lu, Connelly Barnes, and Jiebo Luo. Foreground-aware image inpainting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5840-5848, 2… [cited by applicant]
Chao Yang, Xin Lu, Zhe Lin, Eli Shechtman, Oliver Wang, and Hao Li. High-resolution image inpainting using multi-scale neural patch synthesis. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 201… [cited by applicant]
Jie Yang, Zhiquan Qi, and Yong Shi. Learning to incorporate structure knowledge for image inpainting. InProceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12605-12612, 2020. [cited by applicant]
Zili Yi, Qiang Tang, Shekoofeh Azizi, Daesik Jang, and Zhan Xu. Contextual residual aggregation for ultra high-resolution image inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn… [cited by applicant]
Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Generative image inpainting with contextual attention. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5505… [cited by applicant]
Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Free-form image inpainting with gated convolution. In Proceedings of the IEEE International Conference on Computer Vision, pp. 4471-4480, 2019. [cited by applicant]
Yu Zeng, Zhe Lin, Huchuan Lu, and Vishal M. Patel. Cr-fill: Generative image inpainting with auxiliary contextual reconstruction. InProceedings of the IEEE International Conference on Computer Vision, 2021. [cited by applicant]
Yu Zeng, Zhe Lin, Jimei Yang, Jianming Zhang, Eli Shechtman, and Huchuan Lu. High-resolution image inpainting with iterative confidence feedback and guided upsampling. arXiv preprint arXiv:2005.11742, 2020. [cited by applicant]
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern … [cited by applicant]
Shengyu Zhao, Jonathan Cui, Yilun Sheng, Yue Dong, Xiao Liang, Eric I Chang, and Yan Xu. Large scale image completion via co-modulated generative adversarial networks. arXiv preprint arXiv:2103.10428, 2021. [cited by applicant]
Haitian Zheng, Haofu Liao, Lele Chen, Wei Xiong, Tianlang Chen, and Jiebo Luo. Example-guided image synthesis using masked spatial-channel attention and self-supervision. In European Conference on Computer Vision, pp. 4… [cited by applicant]
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452-1464, 2… [cited by applicant]
Haoming Cai, Jingwen He, Yu Qiao and Chao Dong, “Toward Interactive Modulation for Photo-Realistic Image Restoration,” 2021 IEEE/CV Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Nashville, TN,… [cited by applicant]
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., and Aila, T., “Analyzing and Improving the Image Quality of StyleGAN”, arXiv e-prints, 2019. doi:10.48550/arXiv.1912.04958 (Year: 2019). [cited by applicant]
Kim, H., Choi, Y., Kim, J., Yoo, S., and Uh, Y., “Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing”, arXiv e-prints, 2021. doi:10.48550/arXiv.2104.14754 (Year: 2021). [cited by applicant]
Suvorov, R., “Resolution-robust Large Mask Inpainting with Fourier Convolutions”, arXiv e-prints, 2021. doi: 10.48550/arXiv.2109.07161 (Year: 2021). [cited by applicant]
Xiao, Q., Li, G., and Chen, Q., “Deep Inception Generative Network for Cognitive Image Inpainting”, arXiv e-prints, 2018. doi:10.48550/arXiv.1812.01458 (Year: 2018). [cited by applicant]
Zhao, S., “Large Scale Image Completion via Co-Modulated Generative Adversarial Networks”, arXiv e-prints, 2021. doi: 10.48550/ arXiv.2103.10428 (Year: 2021). [cited by applicant]
Zhu, M., “Image Inpainting by End-to-End Cascaded Refinement With Mask Awareness”, IEEE Transactions on Image Processing, vol. 30, IEEE, pp. 4855-4866, 2021. doi:10.1109/TIP.2021.3076310 (Year: 2021). [cited by applicant]
U.S. Appl. No. 17/661,985, filed Jul. 26, 2024, Notice of Allowance. [cited by applicant]
Hiya Roy, et al. “Image inpainting using frequency-domain priors.” Journal of Electronic Imaging 30.2 (2021): 023016-023016. [cited by applicant]
Rui Xu, et al. “Positional encoding as spatial inductive bias in gans.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. [cited by applicant]
Yu, Yingchen, et al. “Diverse image inpainting with bidirectional and autoregressive transformers.” Proceedings of the 29th ACM International Conference on Multimedia. 2021. [cited by applicant]
Cited By (2)
US 12,367,562 US 12,505,518