IP Library › Granted Patent US 12,499,180
Granted Patent B2
US 12,499,180 · App. 18/045,542 · Granted Dec 16, 2025

Systems and methods for data augmentation

Inventors: Shripad Vilasrao Deshmukh (Maharashtra, IN); Surgan Jandial (Jammu Kashmir, IN); Abhinav Java (Uttar Pradesh, IN); Milan Aggarwal (Delhi, IN); Mausoom Sarkar (New Delhi, IN); Arneh Jain (Kerala, IN); Balaji Krishnamurthy (Uttar Pradesh, IN)
Assignee: ADOBE INC.
G06F18/2411G06F16/285G06F16/35G06F16/353G06F16/55G06F16/906G06F18/2155G06F18/24G06F18/243G06F18/245G06F18/25G06F18/251G06F18/254G06N3/02G06N3/045G06N3/0464G06N3/0475G06N3/08G06N20/00G06T3/4038G06T5/50G06V10/16G06V10/70G06V10/764G06V10/80G06V10/82G06V20/20G06V30/19173G06V30/1918G06V40/172G06F18/2431G06T2207/20084G06T2207/20212G06T2207/20216G06T2207/20221G06T2207/20224G06T2211/00G06T2211/441
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,499,180
App. No.
18/045,542
Granted
Dec 16, 2025
Kind
B2
Abstract

Systems and methods for data augmentation are provided. One aspect of the systems and methods include receiving an image that is misclassified by a classification network; computing an augmentation image based on the image using an augmentation network; and generating an augmented image by combining the image and the augmentation image, wherein the augmented image is correctly classified by the classification network.

Claims (47)

1 . A method for data augmentation comprising:

receiving an image that is classified in a first class and a second class by a classification network;

identifying a first augmentation network corresponding to the first class and a second augmentation network corresponding to the second class;

computing a first augmentation image based on the image using the first augmentation network and computing a second augmentation image based on the image using the second augmentation network; and

generating a first augmented image by combining the image and the first augmentation image using the first augmentation network and generating a second augmented image by combining the image and the second augmentation image using the second augmentation network, wherein the first augmented image is classified in the first class by the classification network and the second augmented image is classified in the second class by the classification network.

2 . The method of claim 1 , further comprising:

classifying the image using the classification network to obtain a false label; and

determining that the image is classified in the first class and the second class based on the false label, wherein the first augmentation image and the second augmentation image are computed based on the determination.

3 . The method of claim 1 , further comprising:

training the classification network based on the first augmented image.

4 . The method of claim 1 , wherein:

the image depicts an object that is classified in the first class and the second class by the classification network, and wherein the first augmented image depicts a first augmented object that is classified in the first class by the classification network and the second augmented image depicts a second augmented object that is classified in the second class by the classification network.

5 . The method of claim 1 , further comprising:

identifying a label for the image; and

selecting the first augmentation network from among a plurality of augmentation networks based on the label.

6 . A non-transitory computer readable medium storing code for data augmentation, the code comprising instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

receiving an image that is classified in a first class and a second class by a classification network;

identifying a first augmentation network corresponding to the first class and a second augmentation network corresponding to the second class;

computing a first augmentation image based on the image using the first augmentation network and computing a second augmentation image based on the image using the second augmentation network; and

generating a first augmented image by combining the image and the first augmentation image using the first augmentation network and generating a second augmented image by combining the image and the second augmentation image using the second augmentation network, wherein the first augmented image is classified in the first class by the classification network and the second augmented image is classified in the second class by the classification network.

7 . The non-transitory computer readable medium of claim 6 , the code further comprising instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:

classifying the image using the classification network to obtain a false label; and

determining that the image is classified in the first class and the second class based on the false label, wherein the first augmentation image and the second augmentation image are computed based on the determination.

8 . The non-transitory computer readable medium of claim 6 , the code further comprising instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:

training the classification network based on the first augmented image.

9 . The non-transitory computer readable medium of claim 6 , wherein:

the image depicts an object that is classified in the first class and the second class by the classification network, and wherein the first augmented image depicts a first augmented object that is classified in the first class by the classification network and the second augmented image depicts a second augmented object that is classified in the second class by the classification network.

10 . The non-transitory computer readable medium of claim 6 , the code further comprising instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:

identifying a label for the image; and

selecting the first augmentation network from among a plurality of augmentation networks based on the label.

11 . A system comprising:

a memory component; and

a processing device coupled to the memory component, the processing device configured to perform operations comprising:

receiving an image that is classified in a first class and a second class by a classification network;

identifying a first augmentation network corresponding to the first class and a second augmentation network corresponding to the second class;

computing a first augmentation image based on the image using the first augmentation network and computing a second augmentation image based on the image using the second augmentation network; and

generating a first augmented image by combining the image and the first augmentation image using the first augmentation network and generating a second augmented image by combining the image and the second augmentation image using the second augmentation network, wherein the first augmented image is classified in the first class by the classification network and the second augmented image is classified in the second class by the classification network.

12 . The system of claim 11 , the processing device further configured to perform operations comprising:

classifying the image using the classification network to obtain a false label; and

determining that the image is classified in the first class and the second class based on the false label, wherein the first augmentation image and the second augmentation image are computed based on the determination.

13 . The system of claim 11 , the processing device further configured to perform operations comprising:

training the classification network based on the first augmented image.

14 . The system of claim 11 , wherein:

the image depicts an object that is classified in the first class and the second class by the classification network, and wherein the first augmented image depicts a first augmented object that is classified in the first class by the classification network and the second augmented image depicts a second augmented object that is classified in the second class by the classification network.

15 . The system of claim 11 , the processing device further configured to perform operations comprising:

identifying a label for the image; and

selecting the first augmentation network from among a plurality of augmentation networks based on the label.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 11, 2022
From: DESHMUKH, SHRIPAD VILASRAO; JANDIAL, SURGAN; JAVA, ABHINAV; AGGARWAL, MILAN; SARKAR, MAUSOOM; JAIN, ARNEH; KRISHNAMURTHY, BALAJI
To: ADOBE INC.
Reel/Frame 061375/0043 →
Continuity (1)
Related Publication 20240119122A1 · Apr 11, 2024
References Cited (42)
US 11610393B2 · Kuen · 2023 [cited by examiner]
US 11861762B2 · Li · 2024 [cited by examiner]
US 20120288186A1 · Kohli · 2012 [cited by examiner]
US 20220004827A1 · Liu · 2022 [cited by examiner]
US 20230316536A1 · Lee · 2023 [cited by examiner]
US 20240320562A1 · Ba · 2024 [cited by examiner]
EP 3582142A1 · 2019 [cited by examiner]
Ullah et al., Design and Evaluation of Object Classifiers for Probabilistic Decision-Making in Autonomous Systems, May 23-27, 2022 [retrieved Nov. 27, 2024], 2022 International Conference on Robotics and Automation (ICR… [cited by examiner]
Chan et al., Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks, Nov. 19, 2019 [retrieved Nov. 27, 2024], Cornell University:arXiv, version: [v1], 26 pages. https://doi.or… [cited by examiner]
Jiangsha et al., Data augmentation by a CycleGAN-based extra-supervised model for nondestructive testing, Jan. 31, 2022 [retrieved Apr. 7, 2025], Measurement Science and Technology, vol. 33, No. 4, 12 pages. DOI 10.1088… [cited by examiner]
Agarwal, et al., “Estimating Example Difficulty using Variance of Gradients”, arXiv preprint arXiv:2008.11600v4 [cs.CV] Jun. 21, 2022, 15 pages. [cited by applicant]
Feng, et al., “A Survey of Data Augmentation Approaches for NLP”, arXiv preprint arXiv:2105.03075v5 [cs.CL] Dec. 1, 2021, 21 pages. [cited by applicant]
Goodfellow, et al., “Explaining and Harnessing Adversarial Examples”, arXiv preprint arXiv:1412.6572v3 [stat. ML] Mar. 20, 2015, 11 pages. [cited by applicant]
He, et al., “Deep Residual Learning for Image Recognition”, arXiv preprint arXiv:1512.03385v1 [cs.CV] Dec. 10, 2015, 12 pages. [cited by applicant]
Ioffe, et al., “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift”, arXiv preprint arXiv:1502.03167v3 [cs.LG] Mar. 2, 2015, 11 pages. [cited by applicant]
Li, et al., “Learning from Noisy Data with Robust Representation Learning”, In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9485-9494, 2021, 10 pages. [cited by applicant]
Liang, et al., “Metashift: A Dataset of Datasets for Evaluating Contextual Distribution Shifts and Training Conflicts”, arXiv preprint arXiv:2202.06523v1 [cs.LG] Feb. 14, 2022, 19 pages. [cited by applicant]
BMa, et al., “Undistillable: Making a Nasty Teacher that CANNOT Teach Students”, arXiv preprint arXiv:2105.07381v1 [cs.LG] May 16, 2021, 12 pages. [cited by applicant]
Ma, et al., “Shufflenet V2: Practical Guidelines for Efficient CNN Architecture Design”, arXiv preprint arXiv:1807.11164v1 [cs.CV] Jul. 30, 2018, 19 pages. [cited by applicant]
Madry, et al., “Towards Deep Learning Models Resistant to Adversarial Attacks”, arXiv preprint arXiv:1706.06083v4 [stat.ML] Sep. 4, 2019, 28 pages. [cited by applicant]
Malinin, et al., “Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks”, arXiv preprint arXiv:2107.07455v3 [cs.LG] Feb. 11, 2022, 44 pages. [cited by applicant]
Mirzadeh, et al., “Improved Knowledge Distillation via Teacher Assistant”, arXiv preprint arXiv:1902.03393v2 [cs.LG] Dec. 17, 2019, 11 pages. [cited by applicant]
Nayak, et al., “DAD: Data-free Adversarial Defense at Test Time”, arXiv preprint arXiv:2204.01568v2 [cs.LG] Apr. 8, 2022, 15 pages. [cited by applicant]
Paiva, et al., “PyHard: a novel tool for generating hardness embeddings to support data-centric analysis”, arXiv preprint arXiv:2109.14430v1 [cs.LG] Sep. 29, 2021, 5 pages. [cited by applicant]
Pulastya, et al., “Assessing the Quality of the Datasets by Identifying Mislabeled Samples”, arXiv preprint arXiv:2109.05000v1 [cs.LG] Sep. 10, 2021, 8 pages. [cited by applicant]
Schwinn, et al., “Exploring Misclassifications of Robust Neural Networks to Enhance Adversarial Attacks”, arXiv preprint arXiv:2105.10304v2 [cs.LG] May 25, 2021, 15 pages. [cited by applicant]
Song, et al., “Learning from Noisy Labels with Deep Neural Networks: A Survey”, arXiv preprint arXiv:2007.08199v7 [cs.LG] Mar. 10, 2022, 19 pages. [cited by applicant]
Su, et al., “Hard Example Guided Hashing for Image Retrieval”, arXiv preprint arXiv:2112.13565v1 [cs.CV] Dec. 27, 2021, 26 pages. [cited by applicant]
Tripathi, et al., “Learning to Generate Synthetic Data via Compositing”, arXiv preprint arXiv:1904.05475v2 [cs.CV] Jul. 8, 2019, 19 pages. [cited by applicant]
Wang, “Data-Free Knowledge Distillation with Soft Targeted Transfer Set Synthesis”, arXiv preprint arXiv:2104.04868v1 [cs.LG] Apr. 10, 2021, 9 pages. [cited by applicant]
Wong, et al., “Fast is Better than Free: Revisiting Adversarial Training”, arXiv preprint arXiv:2001.03994v1 [cs.LG] Jan. 12, 2020, 17 pages. [cited by applicant]
Xiao, et al., “Generating Adversarial Examples with Adversarial Networks”, arXiv preprint arXiv:1801.02610v5 [cs. CR] Feb. 14, 2019, 8 pages. [cited by applicant]
Yang, et al., “Image Data Augmentation for Deep Learning: A Survey”, arXiv preprint arXiv:2204.08610v1 [cs.CV] Apr. 19, 2022, 8 pages. [cited by applicant]
Yoon, et al., “GAIN: Missing Data Imputation using Generative Adversarial Nets”, arXiv preprint arXiv:1806.02920v1 [cs.LG] Jun. 7, 2018, 10 pages. [cited by applicant]
You, et al., “Handling Missing Data with Graph Representation Learning”, arXiv preprint arXiv:2010.16418v1 [cs.LG] Oct. 30, 2020, 15 pages. [cited by applicant]
Zhang, et al., “IDR: Self-Supervised Image Denoising via Iterative Data Refinement”, arXiv preprint arXiv:2111.14358v2 [cs.CV] Mar. 22, 2022, 14 pages. [cited by applicant]
Zhao, “Data Augmentation for Graph Neural Networks”, arXiv preprint arXiv:2006.06830v2 [cs.LG] Dec. 2, 2020, 15 pages. [cited by applicant]
Zhao, et al., “Missing Value Imputation for Mixed Data via Gaussian Copula”, arXiv preprint arXiv:1910.12845v3 [stat.ME] Jun. 16, 2020, 11 pages. [cited by applicant]
Zhao, et al., “CTAB-GAN: Effective Table Data Synthesizing”, arXiv preprint arXiv:2102.08369v2 [cs.LG] May 31, 2021, 11 pages. [cited by applicant]
Zhong, et al., “Random Erasing Data Augmentation”, arXiv preprint arXiv:1708.04896v2 [cs.CV] Nov. 16, 2017, 10 pages. [cited by applicant]
Zhou, et al., “Learning with Noisy Labels via Sparse Regularization”, arXiv preprint arXiv:2108.00192v1 [cs.LG] Jul. 31, 2021, 17 pages. [cited by applicant]
Zhu, et al., “Hard Sample Aware Noise Robust Learning for Histopathology Image Classification”, arXiv preprint arXiv:2112.03694v1 [eess.IV] Dec. 5, 2021, 14 pages. [cited by applicant]