IP Library Granted Patent US 12,511,726
Granted Patent B2
US 12,511,726 · App. 18/344,619 · Granted Dec 30, 2025

System and method for colored artifact detection

Inventors: Matthew John Prokes (Cottage Grove, MN); Ganesh Tiwari (New Brighton, MN)
Assignee: Shutterfly, LLC
G06T7/0002G06T5/77G06T7/70G06T7/90G06F3/04845G06T2200/24G06T2207/10024G06T2207/20092G06T2207/30168
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,511,726
App. No.
18/344,619
Granted
Dec 30, 2025
Kind
B2
Abstract

Methods and systems for detecting colored artifacts on an image of subject are disclosed. An image may be received, wherein the image has undergone background replacement processing. The image may include colored artifacts as a result of errors in background replacement processing. A density of colored artifacts on the image may be determined, and in response to a determination that the image has a density of colored artifacts that exceeds a predetermined threshold, the image may be displayed on a user interface. The user interface may include a selectable element that, in response to being selected, causes the colored artifacts to be highlighted on the image displayed on the user interface. Colored artifacts on the image may then be corrected to improve the image quality before the image is sent to a customer as a final image product.

Claims (55)

1 . A method for detecting colored artifacts on an image of a subject, the method comprising:

receiving an image, wherein the image has undergone background replacement processing;

determining a density of colored artifacts on the image;

in response to a determination that the image has a density of colored artifacts that exceeds a predetermined threshold, displaying the image on a user interface, wherein the user interface includes a selectable element that, in response to being selected, causes the colored artifacts to be highlighted on the image displayed on the user interface;

receiving a selection through the user interface of the selectable element; and

in response to receiving the selection of the selectable element, highlighting the colored artifacts on the image displayed on the user interface.

2 . The method of claim 1 , wherein determining the density of colored artifacts comprises:

receiving an original image, the original image being the image before undergoing background replacement processing;

generating an image mask by comparing the original image and the image to identify pixels that changed during the background replacement processing;

based on the image mask, generating a difference image by identifying, within the identified changed pixels, pixels in a predetermined color range representative of colored artifacts on the image;

identifying clusters of changed pixels in the difference image; and

based on the identified clusters, determining the density of colored artifacts on the image.

3 . The method of claim 1 , wherein the predetermined threshold is a ratio of colored artifacts relative to a surface area the image.

4 . The method of claim 1 , further comprising:

in response to highlighting the colored artifacts on the image, receiving input through the user interface to remove the colored artifacts on the image.

5 . The method of claim 1 , further comprising:

in response to highlighting the colored artifacts, receiving input through the user interface to send the image to a specialist with instructions to remove the colored artifacts on the image.

6 . The method of claim 1 , further comprising:

categorizing the density of colored artifacts on the image as very high, high, medium, or low, wherein the density of colored artifacts exceeds the predetermined threshold based on a categorization of very high or high.

7 . The method of claim 6 , wherein the user interface further includes an indicator of the categorization of the density of colored artifacts on the image.

8 . The method of claim 1 , further comprising,

identifying locations of the colored artifacts on the image with (x,y) coordinates.

9 . The method of claim 1 , further comprising:

receiving input through the user interface indicating that a quality of the image is acceptable to send the image to a customer based on a manual inspection for colored artifacts on the image.

10 . The method of claim 1 , wherein the highlighting of the colored artifacts is presented on the user interface in a color different from the colored artifacts.

11 . The method of claim 1 , determining a density of colored artifacts on the image is performed on a defined portion of the image, the defined portion being less than the entire image.

12 . The method of claim 1 , wherein the colored artifacts are blue artifacts.

13 . A system for detecting colored artifacts on an image of a subject, the system comprising:

a user interface; and

a computing system including a processor, and a memory communicatively coupled to the processor, the memory storing instructions executable by the processor to:

receive an image, wherein the image has undergone background replacement processing;

determine a density of colored artifacts on the image;

in response to a determination that the image has a density of colored artifacts that exceeds a predetermined threshold, display the image on a user interface, wherein the user interface includes a selectable element that, in response to being selected, causes the colored artifacts to be highlighted on the image displayed on the user interface;

receive a selection through the user interface of the selectable element; and

in response to receiving the selection of the selectable element, highlight the colored artifacts on the image displayed on the user interface.

14 . The system of claim 13 , wherein to determine a density of colored artifacts on the image, the instructions stored in the memory are further executable by the process to:

receive an original image, the original image being the image before undergoing background replacement processing;

generate an image mask by comparing the original image and the image to identify pixels that changed during the background replacement processing;

based on the image mask, generate a difference image by identifying, within the identified changed pixels, pixels in a predetermined color range representative of colored artifacts on the image;

identify clusters of changed pixels in the difference image; and

based on the identified clusters, determine the density of colored artifacts on the image.

15 . The system of claim 13 , wherein the instructions stored in the memory are further executable by the process to:

in response to highlighting the colored artifacts on the image, receive input through the user interface to remove the colored artifacts on the image.

16 . The system of claim 13 , wherein the instructions stored in the memory are further executable by the process to:

categorize the density of colored artifacts on the image as very high, high, medium, or low, wherein the density of colored artifacts exceeds the predetermined threshold based on a categorization of very high or high.

17 . The system of claim 16 , wherein the user interface further includes an indicator of the categorization of the density of colored artifacts on the image.

18 . The system of claim 13 , wherein the instructions stored in the memory are further executable by the process to:

identify locations of the colored artifacts on the image with (x,y) coordinates.

19 . The system of claim 13 , wherein the instructions stored in the memory are further executable by the process to:

receive input through the user interface indicating that a quality of the image is acceptable to send the image to a customer based on a manual inspection for colored artifacts on the image.

20 . A computer-readable storage device storing data instructions that, when executed by a processing device of a computing device, cause the computing device to generate a user interface comprising:

an image of a subject, wherein the image has undergone background replacement processing and includes a density of colored artifacts that exceeds a predetermined threshold;

an indicator reflecting a level of colored artifacts detected on the image;

a first selectable element that, in response to being selected, causes the colored artifacts on the image to be highlighted, wherein the highlighting of the colored artifacts is presented in a color different from the colored artifacts; and

a second selectable element that, in response to being selected, causes a notification to be sent to a photography retailer computing device indicating that a quality of the image is acceptable to send the image to a customer computing device based on a manual inspection for colored artifacts on the image.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 3, 2025
From: PROKES, MATTHEW JOHN; TIWARI, GANESH
To: SHUTTERFLY, LLC
Reel/Frame 073103/0741 →
SECURITY INTEREST Recorded Jun 20, 2024
From: SHUTTERFLY, LLC
To: WILMINGTON TRUST, NATIONAL ASSOCIATION
Reel/Frame 067788/0387 →
Continuity (1)
Related Publication 20250005729A1 · Jan 2, 2025
References Cited (27)
US 6912313B2 · Li · 2005 [cited by applicant]
US 8319797B2 · Swanson et al. · 2012 [cited by applicant]
US 8730341B2 · Nashizawa · 2014 [cited by applicant]
US 8861867B2 · Bvn · 2014 [cited by applicant]
US 8878958B2 · Ichiyama et al. · 2014 [cited by applicant]
US 9838661B2 · Benson · 2017 [cited by applicant]
US 10474006B2 · Ariav et al. · 2019 [cited by applicant]
US 10997690B2 · Schwartz et al. · 2021 [cited by applicant]
US 20030012435A1 · Forde · 2003 [cited by applicant]
US 20130004446A1 · Bui et al. · 2013 [cited by applicant]
US 20130024324A1 · Davis, Jr. · 2013 [cited by applicant]
US 20140085509A1 · Ichikawa · 2014 [cited by examiner]
US 20150277809A1 · Kim · 2015 [cited by examiner]
US 20190154822A1 · Berlin · 2019 [cited by examiner]
US 20210319781A1 · Gullo · 2021 [cited by examiner]
US 20220005190A1 · N G · 2022 [cited by examiner]
US 20220375426A1 · Nam et al. · 2022 [cited by applicant]
US 20220377259A1 · Li · 2022 [cited by examiner]
US 20230352147A1 · Abdolell · 2023 [cited by examiner]
US 20250005729A1 · Prokes · 2025 [cited by examiner]
US 20250173207A1 · Nannapaneni · 2025 [cited by examiner]
JP 2011041094A · 2011 [cited by applicant]
Naftali Harris; https://www.naftaliharris.com/blog/visualizing-dbscan-clustering; Visualizing DBSCAN Clustering; Jan. 24, 2015; 3 pages. [cited by applicant]
The Apache Software Foundation; https://commons.apache.org/proper/commons-math/userguide/ml.html; 16 Machine Learning; Dec. 22, 2022; 4 pages. [cited by applicant]
The Apache Software Foundation; https://commons.apache.org/proper/commons-math/javadocs/api-3.6.1/org/apache/commons/math3/ml/clustering/DBSCANClusterer.html; Class DBSCANClusterer<T extends Clusterable>; copyright 2003… [cited by applicant]
36PIX Inc .; Smart technology to create perfect images; copyright 2023; accessed on Jun. 29, 2023; 20 pages. [cited by applicant]
REMOVEBG; www.remove.bg; Remove Image Background; accessed on Jun. 29, 2023; 12 pages. [cited by applicant]