IP Library Granted Patent US 10,671,920
Granted Patent B2
US 10,671,920 · App. 15/938,897 · Granted Jun 2, 2020

Generative augmentation of image data

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,671,920
App. No.
15/938,897
Granted
Jun 2, 2020
Kind
B2
Abstract

Systems and methods to receive one or more first images associated with a training set of images to train a machine learning model; provide the one or more first images as a first input to a first set of layers of computational units, wherein the first set of layers utilizes image filters; provide a first output of the first set of layers of computational units as a second input to a second layer of the computational units, wherein the second layer utilizes random parameter sets for computations; obtain distortion parameters from the second layer of the computational units; generate one or more second images comprising a representation of the one or more first images modified with the distortion parameters; obtain, as a third output, the one or more second images; and add the one or more second images to the training set of images to train the machine learning model.

Claims (64)

1. A method comprising:

receiving, by a processing device, one or more first images associated with a training set of images to train a machine learning model in training;

providing, by the processing device, the one or more first images as a first input to a first set of layers of computational units, wherein the first set of layers utilizes image filters;

providing a first output of the first set of layers of computational units as a second input to a second layer of the computational units, wherein the second layer utilizes random parameter sets for computations;

obtaining distortion parameters from the second layer of the computational units;

generating one or more second images based on the one or more first images and the distortion parameters;

obtaining, as a third output, the one or more second images; and

adding the one or more second images to the training set of images to train the machine learning model.

2. The method of claim 1 , further comprising:

providing a second output of the second layer of the computational units as a third input to a third set of layers of the computational units.

3. The method of claim 1 , wherein the machine learning model comprises a convolutional neural network.

4. The method of claim 1 , wherein providing the first input comprises:

dividing the one or more first images into a plurality of portions; and

providing each of the plurality of portions to the first set of layers of the computational units.

5. The method of claim 4 , wherein obtaining the distortion parameters comprises:

obtaining a distortion parameter for each of the plurality of portions of the one or more first images.

6. The method of claim 1 , further comprising:

generating a randomized kernel matrix for the second layer of the computational units based on one or more matrices.

7. The method of claim 6 , wherein the one or more matrices comprise one or more of:

a first matrix of mean values, the first matrix initialized with random values;

a second matrix of standard deviation values, the second matrix initialized with zero values;

a third matrix of displacement values, the third matrix based on a number of filters to apply to the second input; or

a fourth matrix that is based on an arbitrary standard deviation value and a normal distribution value.

8. The method of claim 7 , wherein the arbitrary standard deviation value specifies a roughness of image distortions.

9. The method of claim 6 , wherein parameters for the one or more matrices comprise at least one of filter height, filter width, image height, image width, size of filter, number of channels, number of filters, or number of images.

10. The method of claim 9 , wherein the image height and image width each comprises arbitrary values.

11. The method of claim 1 , wherein the one or more first images comprise one or more of:

one or more hieroglyphs;

one or more Chinese-Japanese-Korean (CJK) symbols;

one or more Arabic strings; or

a combination of one or more other symbols.

12. The method of claim 1 , wherein generating the one or more second images comprises:

generating the one or more second images corresponding to naturally distorted images.

13. The method of claim 1 , wherein generating the one or more second images further comprises performing convolution prior to obtaining the one or more second images as the third output.

14. The method of claim 7 , wherein the first matrix, the second matrix, and the third matrix each comprises learnable parameters.

15. A system comprising:

a memory; and

a processor, coupled to the memory, the processor to:

receive one or more first images associated with a training set of images to train a machine learning model in training;

provide the one or more first images as a first input to a first set of layers of computational units, wherein the first set of layers utilizes image filters;

provide a first output of the first set of layers of computational units as a second input to a second layer of the computational units, wherein the second layer utilizes random parameter sets for computations;

obtain distortion parameters from the second layer of the computational units;

provide a second output of the second layer of the computational units as a third input to a third set of layers of the computational units;

provide a third output of the third set of layers of the computational units as one or more second images, the third output being based on the one or more first images and the distortion parameters; and

add the one or more second images to the training set of images to train the machine learning model.

16. The system of claim 15 , wherein the processor is further to:

generate a randomized kernel matrix for the second layer of the computational units based on one or more matrices.

17. The system of claim 16 , wherein the one or more matrices comprise one or more of:

a first matrix of mean values, the first matrix initialized with random values;

a second matrix of standard deviation values, the second matrix initialized with zero values;

a third matrix of displacement values, the third matrix based on a number of filters to apply to the second input; or

a fourth matrix that is based on an arbitrary standard deviation value and a normal distribution value.

18. The system of claim 17 , wherein the arbitrary standard deviation value specifies roughness of image distortions.

19. A computer-readable non-transitory storage medium comprising executable instructions that, when executed by a processing device, cause the processing device to:

receive one or more first images associated with a training set of images to train a machine learning model in training;

provide the one or more first images as a first input to a first set of layers of computational units, wherein the first set of layers utilizes image filters;

provide a first output of the first set of layers of computational units as a second input to a second layer of the computational units, wherein the second layer utilizes random parameter sets for computations;

obtain distortion parameters from the second layer of the computational units;

generate one or more second images comprising a representation of the one or more first images modified with the distortion parameters;

obtain, as a third output, the one or more second images; and

add the one or more second images to the training set of images to train the machine learning model.

20. The computer-readable non-transitory storage medium of claim 19 , wherein to provide the first input, the processing device is to:

divide the one or more first images into a plurality of portions; and

provide each of the plurality of portions to the first set of layers of the computational units.

Assignments (3)
SECURITY INTEREST Recorded Aug 14, 2023
From: ABBYY INC.; ABBYY USA SOFTWARE HOUSE INC.; ABBYY DEVELOPMENT INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 064730/0964 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 25, 2022
From: ABBYY PRODUCTION LLC
To: ABBYY DEVELOPMENT INC.
Reel/Frame 059249/0873 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 29, 2018
From: ZUEV, KONSTANTIN; SAUTINS, ANDREJS
To: ABBYY PRODUCTION LLC
Reel/Frame 045386/0215 →