IP Library › Granted Patent US 11,651,480
Granted Patent B2
US 11,651,480 · App. 16/951,908 · Granted May 16, 2023

Systems and methods for selective enhancement of objects in images

Inventor: Dmitry Sytnik (Kyiv, UA)
Assignee: Shinyfields Limited
G06T5/30G06F3/0482G06F3/04845G06T5/002G06T5/009G06T5/50G06T2207/10024G06T2207/20084G06T2207/20212
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,651,480
App. No.
16/951,908
Granted
May 16, 2023
Kind
B2
Abstract

Exemplary embodiments are directed to a system for selective enhancement of an object in an image. The system includes an interface configured to receive as input an original image, and a processing device in communication with the interface. The processing device is configured to process the original image using a neural network to detect one or more objects in the original image, generate a neural network mask of the original image for the one or more objects in the original image, apply one or more enhancements to the objects associated with the neural network mask, the one or more modules generating an enhanced image including the one or more enhancements to the objects, and generate a combined image, the combined image including the original image combined with the one or more enhancements to the objects of the enhanced image.

Claims (50)

1. A system for selective enhancement of an object in an image, the system comprising:

an interface configured to receive as input an original image; and

a processing device in communication with the interface, the processing device configured to:

(i) process the original image using a neural network to detect one or more objects in the original image, wherein the original image includes a foreground and a background, and wherein the one or more objects include the background of the original image;

(ii) generate a neural network mask of the original image for the one or more objects in the original image;

(iii) apply one or more enhancements to the objects associated with the neural network mask, the one or more modules generating an enhanced image including the one or more enhancements to the objects; and

(iv) generate a combined image, the combined image including the original image combined with the one or more enhancements to the objects of the enhanced image;

wherein the processing device is further configured to:

generate a refined mask for each pixel of the original image associated with the background; and

generate a dilated mask, the dilated mask including a dilation or indentation from a border extending between the foreground and the background.

2. The system of claim 1 , wherein the one or more objects include a sky in the original image.

3. The system of claim 2 , wherein the processing device determines a category type of the sky in the original image.

4. The system of claim 3 , wherein the category type includes at least one of a blue day sky, a sunset sky, a dawn sky, a cloudy sky, a stormy sky, a night sky, or a foggy sky.

5. The system of claim 1 , wherein the processing device generates a blurred mask, the blurred mask including a blur along a border extending between the foreground and the background.

6. The system of claim 1 , wherein the one or more enhancements applied to the objects associated with the neural network mask are applied to the background independently from the foreground of the original image.

7. The system of claim 1 , wherein the one or more enhancements applied to the objects associated with the neural network mask are applied to the background without affecting the foreground of the original image.

8. A system for selective enhancement of an object in an image, the system comprising:

an interface configured to receive as input an original image; and

a processing device in communication with the interface, the processing device configured to:

(i) process the original image using a neural network to detect one or more objects in the original image;

(ii) generate a neural network mask of the original image for the one or more objects in the original image;

(iii) apply one or more enhancements to the objects associated with the neural network mask, the one or more modules generating an enhanced image including the one or more enhancements to the objects; and

(iv) generate a combined image, the combined image including the original image combined with the one or more enhancements to the objects of the enhanced image;

wherein applying the one or more enhancements includes:

generating a brightness corrected image including brightness enhancement of the original image; and

generating a contrast corrected image including contrast enhancement of the original image.

9. The system of claim 1 , wherein the processing device generates a saturation corrected image including saturation enhancement of the original image.

10. The system of claim 1 , wherein the processing device generates a color improved image including color enhancement of the original image.

11. The system of claim 1 , wherein the processing device generates a reduced noise image by reducing noise associated with a night sky in the original image.

12. A system for selective enhancement of an object in an image, the system comprising:

an interface configured to receive as input an original image; and

a processing device in communication with the interface, the processing device configured to:

(i) process the original image using a neural network to detect one or more objects in the original image;

(ii) generate a neural network mask of the original image for the one or more objects in the original image;

(iii) apply one or more enhancements to the objects associated with the neural network mask, the one or more modules generating an enhanced image including the one or more enhancements to the objects; and

(iv) generate a combined image, the combined image including the original image combined with the one or more enhancements to the objects of the enhanced image;

wherein the processing device determines whether pixels in the neural network mask have a luminosity value above, below or equal to a luminosity threshold value; and

wherein:

if the luminosity value of a pixel in the neural network mask is equal to or above the luminosity threshold value, the one or more enhancements are fully applied to the pixel; and

if the luminosity value of the pixel in the neural network mask is below the luminosity threshold value, the one or more enhancements are not applied to the pixel or are applied at a reduced strength to the pixel.

13. A system for selective enhancement of an object in an image, the system comprising:

an interface configured to receive as input an original image; and

a processing device in communication with the interface, the processing device configured to:

(i) process the original image using a neural network to detect one or more objects in the original image;

(ii) generate a neural network mask of the original image for the one or more objects in the original image;

(iii) apply one or more enhancements to the objects associated with the neural network mask, the one or more modules generating an enhanced image including the one or more enhancements to the objects; and

(iv) generate a combined image, the combined image including the original image combined with the one or more enhancements to the objects of the enhanced image;

wherein:

the interface includes an image selection section with the combined image and one or more additional original images; and

the interface includes a first submenu for selecting the combined image and copying the one or more enhancements applied to the combined image, and the interface includes a second submenu for selecting one or more of the additional original images and applying the copied one or more enhancements of the combined image to the selected one or more of the additional original images.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 18, 2020
From: SYTNIK, DMITRY
To: SHINYFIELDS LIMITED
Reel/Frame 054412/0197 →
Continuity (2)
Provisional Application 62936856 · Nov 18, 2019
Related Publication 20210150681A1 · May 20, 2021