IP Library Granted Patent US 11,182,877
Granted Patent B2
US 11,182,877 · App. 16/534,460 · Granted Nov 23, 2021

Techniques for controlled generation of training data for machine learning enabled image enhancement

Inventors: Bo Zhu (Charlestown, MA); Haitao Yang (Boston, MA); Liying Shen (Charlestown, MA)
Assignee: BlinkAI Technologies, Inc.
G06T5/002G06K9/46G06K9/6256G06N3/04G06N20/00G06T7/11G06T7/44G06T2207/10016G06T2207/20132
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,182,877
App. No.
16/534,460
Granted
Nov 23, 2021
Kind
B2
Abstract

Described herein are systems and techniques for generating training data for use in training a machine learning model for image enhancement. The system may access a target image of a displayed video frame, wherein the target image represents a target output of the machine learning model. The system may access an input image of the displayed video frame, wherein the input image corresponds to the target image and represents an input to the machine learning model. The system may train the machine learning model using the target image and the input image corresponding to the target image to obtain a trained machine learning model.

Claims (47)

1. A method of training a machine learning model for enhancing images, the method comprising:

using at least one computer hardware processor to perform:

accessing a target image of a displayed video frame, wherein the target image represents a target output of the machine learning model;

accessing an input image of the displayed video frame, wherein the input image corresponds to the target image and represents an input to the machine learning model; and

training the machine learning model using the target image and the input image corresponding to the target image to obtain a trained machine learning model.

2. The method of claim 1 , further comprising:

capturing, using an imaging device, the target image of the displayed video frame using a first exposure time; and

capturing, using the imaging device, the input image of the displayed video frame using a second exposure time, wherein the second exposure time is less than the first exposure time.

3. The method of claim 1 , further comprising:

capturing, using an imaging device, the input image of the displayed video frame with a neutral density filter; and

capturing, using the imaging device, the target image of the displayed video frame without a neutral density filter.

4. The method of claim 1 , further comprising:

capturing, using an imaging device, the input image of the displayed video frame; and

capturing, using the imaging device, the target image of the displayed video frame by averaging each pixel location of multiple stationary captures of the video frame.

5. The method of claim 1 , further comprising:

capturing, using an imaging device, the target image of the displayed video frame using a first exposure time, wherein the displayed video frame is displayed at a first brightness; and

capturing, using the imaging device, the input image of the displayed video frame using the first exposure time, wherein the displayed video frame is displayed at a second brightness darker than the first brightness.

6. The method if claim 1 ,

wherein the input image and the target image each comprise the displayed video frame at an associated inner portion, such that the input image and target image include second data different than the data associated with the displayed video frame; and

the method further comprises cropping each of the input image and the target image to include the first data and to exclude the second data.

7. The method of claim 6 , wherein the input image and the target image each comprise a same first number of pixels that is less than a second number of pixels of the display device displaying the video frame.

8. The method of claim 1 , further comprising:

accessing an image;

providing the image as input to the trained machine learning model to obtain a corresponding output indicating updated pixel values for the image; and

updating the image using the output from the trained machine learning model.

9. The method of claim 1 , further comprising:

accessing a plurality of:

additional target images, wherein each target image of the additional target images:

is of an associated displayed video frame; and

represents an associated target output of the machine learning model for the associated displayed video frame; and

additional input images, wherein each input image of the additional input images:

corresponds to a target image of the additional target images, such that the input image is of the same displayed video frame as the corresponding target image; and

represents an input to the machine learning model for the corresponding target image; and

training the machine learning model using (a) the target image and the input image corresponding to the target image, and (b) the plurality of additional target images and the plurality of additional associated input images, to obtain a trained machine learning model.

10. A system for training a machine learning model for enhancing images, the system comprising:

a display for displaying a video frame of a video;

a digital imaging device configured to:

capture a target image of the displayed video frame, wherein the target image represents a target output of the machine learning model; and

capture an input image of the displayed video frame, wherein the input image corresponds to the target image and represents an input to the machine learning model; and

a computing device comprising at least one hardware processor and at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform:

accessing the target image and the input image; and

training the machine learning model using the target image and the input image corresponding to the target image to obtain a trained machine learning model.

11. The system of claim 10 , wherein the display comprises a television, a projector, or some combination thereof.

12. At least one non-transitory computer readable storage medium storing processor-executable instructions that, when executed by at least one processor, cause the at least one processor to perform:

accessing a target image of a displayed video frame, wherein the target image represents a target output of a machine learning model;

accessing an input image of the displayed video frame, wherein the input image corresponds to the target image and represents an input to the machine learning model; and

training the machine learning model using the target image and the input image corresponding to the target image to obtain a trained machine learning model.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 9, 2022
From: BLINKAI TECHNOLOGIES, INC.
To: META PLATFORMS, INC.
Reel/Frame 061908/0188 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2022
From: BLINKAI TECHNOLOGIES, INC.
To: META PLATFORMS, INC.
Reel/Frame 059237/0689 →
CHANGE OF NAME Recorded Dec 30, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058600/0190 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 21, 2020
From: SHEN, LIYING; ZHU, BO; YANG, HAITAO
To: BLINKAI TECHNOLOGIES, INC.
Reel/Frame 054714/0662 →
Cited By (1)
US 12,231,790