IP Library › Granted Patent US 11,657,513
Granted Patent B2
US 11,657,513 · App. 17/095,466 · Granted May 23, 2023

Method and system for generating a tri-map for image matting

Inventors: Avinav Goel (Bangalore, IN); Jagadeesh Kumar Malla (Bangalore, IN); Mahesh Putane Jagadeeshrao (Bangalore, IN); Manoj Kumar (Bangalore, IN); Pavan Sudheendra (Bangalore, IN); Tanushree Gupta (Bangalore, IN)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G06T7/194G06N3/08G06T7/11G06T7/155G06T2207/20036G06T2207/20081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,657,513
App. No.
17/095,466
Granted
May 23, 2023
Kind
B2
Abstract

A system and a method of performing an image matting on an image are provided. The method includes detecting, by an image processing system, one or more objects in the image; determining, by the image processing system, a confidence map associated with the image using one or more image segmentation techniques for each of the one or more objects; and generating, by the image processing system, a tri-map for each of the one or more objects in the image from the confidence map based on at least one of a size of each of the one or more objects in the image and a distance between a first pixel in the image and a second pixel in at least one of the one or more objects in the image, wherein the tri-map is used to perform the image matting.

Claims (39)

1. A method of performing an image matting on an image, the method comprising:

detecting, by an image processing system, one or more objects in the image;

determining, by the image processing system, a confidence map associated with the image using one or more image segmentation techniques for each of the one or more objects; and

generating, by the image processing system, a tri-map for each of the one or more objects in the image from the confidence map based on a distance between a first pixel in the image and a second pixel in an at least one of the one or more objects in the image,

wherein the generating the tri-map for each of the one or more objects in the image further comprises:

comparing the distance between the first pixel in the image and the second pixel in the at least one of the one or more objects in the image with a transformation value corresponding to each of the one or more objects; and

labelling the first pixel as an unknown pixel based on the distance being less than the transformation value, or labelling the first pixel as at least one of a foreground pixel or a background pixel using the confidence map based on the distance being greater than or equal to the transformation value, and

wherein the tri-map is used to perform the image matting.

2. The method according to claim 1 , wherein the detecting the one or more objects in the image comprises identifying the one or more objects in the image using one or more object detection techniques.

3. The method according to claim 2 , wherein the one or more object detection techniques comprise at least one of a bounding box technique using machine learning based viola-j ones object detection framework using haar features, a scale-invariant feature transform (SIFT), a histogram of oriented gradient (HOG) features, a deep learning based region proposal, a single-shot refinement neural network for object detection, or a single-shot multi-box detector.

4. The method according to claim 1 , wherein the determining the confidence map comprises:

computing the confidence map for the one or more objects in the image based on at least one of a pre-trained deep learning model, a depth sensor, or a Time of Flight (TOF) sensor; and

modifying the confidence map using a color similarity based boundary correction technique.

5. The method according to claim 1 , wherein the performing the image matting comprises:

providing the image and the tri-map of the image to a first pre-trained deep learning model; and

generating an alpha matte of the image,

wherein a foreground and a background in the image is separated using the alpha matte.

6. The method according to claim 1 , wherein the second pixel in the at least one of the one or more objects in the image is determined based on a bounding box applied to each of the one or more objects in the image, and

wherein the second pixel is located at a center of the bounding box.

7. The method according to claim 1 , wherein the generating the tri-map for each of the one or more objects in the image comprises generating the tri-map using morphological operations, and

wherein kernel size of the morphological operations are determined based on a ratio of a size of the image to the size of each of the one or more objects in the image.

8. An image processing system for performing an image matting on an image, the image processing system comprising:

a memory; and

a processor configured to:

detect one or more objects in the image;

determine a confidence map associated with the image using one or more image segmentation techniques for each of the one or more objects; and

generate a tri-map for each of the one or more objects in the image from the confidence map based on a distance between a first pixel in the image and a second pixel in an at least one of the one or more objects in the image,

wherein the processor is further configured to:

compare the distance between the first pixel in the image and the second pixel in the at least one of the one or more objects in the image with a transformation value corresponding to each of the one or more objects; and

label the first pixel as an unknown pixel based on the distance being less than the transformation value, or label the first pixel as at least one of a foreground pixel or a background pixel using the confidence map based on the distance being greater than or equal to the transformation value,

wherein the tri-map is used to perform the image matting.

9. The image processing system according to claim 8 , wherein the processor is further configured to identify the one or more objects in the image using one or more object detection techniques.

10. The image processing system according to claim 8 , wherein the processor is further configured to:

compute the confidence map for the one or more objects in the image based on at least one of a pre-trained deep learning model, a depth sensor, or a Time of Flight (TOF) sensor; and

modify the confidence map using a color similarity based boundary correction technique.

11. The image processing system according to claim 8 , wherein the processor is further configured to:

provide the image and the tri-map of the image to a first pre-trained deep learning model; and

generate an alpha matte of the image,

wherein a foreground and a background in the image is separated using the alpha matte.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 11, 2020
From: GOEL, AVINAV; KUMAR MALLA, JAGADEESH; PUTANE JAGADEESHRAO, MAHESH; KUMAR, MANOJ; SUDHEENDRA, PAVAN; GUPTA, TANUSHREE
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 054339/0882 →
Priority Claims (2)
IN 201941049053 · Nov 29, 2019 · national
IN 201941049053 · Sep 30, 2020 · national
Continuity (1)
Related Publication 20210166400A1 · Jun 3, 2021