IP Library › Granted Patent US 12,495,947
Granted Patent B2
US 12,495,947 · App. 18/185,194 · Granted Dec 16, 2025

Control of cleaning machine cycles using machine vision

Inventors: Conor Sylvester Smith (Saint Louis Park, MN); Paul Dominic Christian (Apple Valley, MN); Elizabeth Minhee Han (St. Paul, MN); Paul R. Kraus (Apple Valley, MN); Rachel Marie McGinness (Rosemount, MN)
Assignee: Ecolab USA Inc.
A47L15/4295A47L15/0026A47L15/0028A47L15/0047A47L15/46A47L2401/04A47L2501/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,495,947
App. No.
18/185,194
Granted
Dec 16, 2025
Kind
B2
Abstract

Image classification is used to dynamically control one or more wash parameters in an automated cleaning machine. An imaging device installed in the wash chamber of the cleaning machine captures one or more image(s) of articles to be cleaned at various times throughout a cleaning process. A computing device analyzes the captured image(s) to classify the images as to an article type and a rack volume. Based on the article type classification and the rack volume, the computing device dynamically controls one or more parameters of the cleaning process to achieve a satisfactory cleaning and/or sanitizing result.

Claims (41)

1 . A system comprising:

an imaging device configured to:

capture a first image of first articles that are in a wash chamber of a cleaning machine for cleaning while the first articles are in the wash chamber for a first cleaning process performed by the cleaning machine; and

capture a second image of second articles that are in the wash chamber of the cleaning machine for cleaning while the second articles are in the wash chamber for a second cleaning process performed by the cleaning machine;

a sump temperature sensor configured to measure a temperature of a solution in a sump of the cleaning machine; and

a computing device comprising at least one processor configured to:

determine that a rack in the wash chamber is a rewashed rack, wherein the at least one processor is configured to determine that the rack is the rewashed rack based on: (i) a comparison of the first image and the second image to determine whether the second articles are identical to the first articles, wherein, as part of determining whether the second articles are identical to the first articles, the at least one processor determines based on the first image and the second image whether the first articles and the second articles are rearranged within the rack and (ii) based on a sump temperature difference for the second cleaning process being less than a sump temperature difference for the first cleaning process, wherein the sump temperature difference for the second cleaning process is a difference between a first and a second temperature of solution in the sump measured by the sump temperature sensor during the second cleaning process, and the sump temperature difference for the first cleaning process is a difference between a third and a fourth temperature of solution in the sump measured by the sump temperature sensor during the first cleaning process; and

adjust, based on the determination that the rack is a rewashed rack, one or more of a wash cycle time or a rinse cycle time performed by the cleaning machine during execution of the second cleaning process.

2 . The system of claim 1 , wherein the at least one processor is further configured to determine a rewash frequency statistic based on a total number of rewashed racks within a specified period of time and a total number of racks washed within the specified period of time.

3 . The system of claim 1 , wherein:

the at least one processor is configured to, as part of determining that the rack is the rewashed rack:

compare at least one of an article type or a rack volume determined for the second cleaning process with at least one of an article type or a rack volume, respectively, determined for the first cleaning process; and

determine that the second articles are identical to the first articles based on the comparison of the at least one the article type or the rack volume determined for the second cleaning process with at least one of the article type or the rack volume, respectively, determined for the first cleaning process.

4 . The system of claim 1 , wherein:

the at least one processor is configured to determine that the rack is the rewashed rack based on one or more of an article type or rack volume over a course of the first cleaning process and the second cleaning process.

5 . The system of claim 1 , wherein the imaging device is configured to capture the first image before a start of the first cleaning process and to capture the second image before a start of the second cleaning process.

6 . The system of claim 1 , wherein:

the imaging device is configured to capture the first image during a dwell time between a wash cycle and a rinse cycle of the first cleaning process, and

the imaging device is configured to capture the second image during a dwell time between a wash cycle and a rinse cycle of the second cleaning process.

7 . The system of claim 1 , wherein:

the imaging device is configured to capture the first image after completion of the first cleaning process, and

the imaging device is configured to capture the second image before a start of the second cleaning process.

8 . A method comprising:

capturing a first image of first articles that are in a wash chamber of a cleaning machine for cleaning while the first articles are in the wash chamber for a first cleaning process performed by the cleaning machine;

capturing a second image of second articles that are in the wash chamber of the cleaning machine for cleaning while the second articles are in the wash chamber for a second cleaning process performed by the cleaning machine;

determining, by at least one processor of a computing device, that a rack in the wash chamber is a rewashed rack, wherein determining that the rack is the rewashed rack comprises determining that the rack is the rewashed rack based on: (1) a comparison of the first image and the second image to determine whether the second articles are identical to the first articles, wherein, determining whether the second articles are identical to the first articles comprises determining based on the first image and the second image whether the first articles and the second articles are rearranged within the rack and (ii) a sump temperature difference for the second cleaning process being less than a sump temperature difference for the first cleaning process, wherein the sump temperature difference for the second cleaning process is a difference between a first and a second temperature of a solution in a sump of the cleaning machine measured by a sump temperature sensor during the second cleaning process, and the sump temperature difference for the first cleaning process is a difference between a third and a fourth temperature of solution in the sump measured by the sump temperature sensor during the first cleaning process;

adjusting, by the at least one processor, based on the determination that the rack is the rewashed rack, one or more of a wash cycle time or a rinse cycle time performed by the cleaning machine during execution of the second cleaning process.

9 . The method of claim 8 , further comprising determining, by the at least one processor, a rewash frequency statistic based on a total number of rewashed racks within a specified period time and a total number of racks washed within the specified period of time.

10 . The method of claim 8 , further comprising:

comparing, by the at least one processor, at least one of an article type or a rack volume determined for the second cleaning process with at least one of an article type or a rack volume, respectively, determined for the first cleaning process; and

determining, by the at least one processor, whether the second articles are identical to the first articles based on the comparison of the at least one the article type or the rack volume determined for the second cleaning process with at least one of the article type or the rack volume, respectively, determined for the first cleaning process.

11 . The method of claim 8 , further comprising initiating, by the at least one processor, the second cleaning process after adjusting at least one of the wash cycle time or the rinse cycle time for the second cleaning process.

12 . The method of claim 8 , wherein:

the method further comprises: determining, by the at least one processor, that the rack is the rewashed rack based on one or more of an article type or rack volume over a course of the first cleaning process and the second cleaning process.

13 . The method of claim 8 , wherein the first image is captured before a start of the first cleaning process and the second image is captured before a start of the second cleaning process.

14 . The method of claim 8 , wherein:

the first image is captured during a dwell time between a wash cycle and a rinse cycle of the first cleaning process, and

the second image is captured during a dwell time between a wash cycle and a rinse cycle of the second cleaning process.

15 . The method of claim 8 , wherein:

the first image is captured after completion of the first cleaning process, and

the second image is captured before a start of the second cleaning process.

Continuity (3)
Continuation 17018363 · Sep 11, 2020
Provisional Application 62899592 · Sep 12, 2019
Related Publication 20230218137A1 · Jul 13, 2023
References Cited (111)
US 4735219A · Seeland · 1988 [cited by applicant]
US 6615850B1 · Hornung · 2003 [cited by applicant]
US 6622754B1 · Roth et al. · 2003 [cited by applicant]
US 7695568B2 · Gaus · 2010 [cited by applicant]
US 9289107B2 · Ellingson et al. · 2016 [cited by applicant]
US 11627861B2 · Smith et al. · 2023 [cited by applicant]
US 11666198B2 · Han et al. · 2023 [cited by applicant]
US 20070181162A1 · Classen et al. · 2007 [cited by applicant]
US 20090151751A1 · Bragg · 2009 [cited by applicant]
US 20100205819A1 · Ashrafzadeh et al. · 2010 [cited by applicant]
US 20110108073A1 · Tameishi · 2011 [cited by applicant]
US 20110209729A1 · Beaudet et al. · 2011 [cited by applicant]
US 20120138092A1 · Ashrafzadeh et al. · 2012 [cited by applicant]
US 20140041688A1 · Maennle et al. · 2014 [cited by applicant]
US 20140041695A1 · Ellingson · 2014 [cited by examiner]
US 20160324396A1 · Hong · 2016 [cited by examiner]
US 20170119232A1 · Ugel · 2017 [cited by examiner]
US 20180092508A1 · Thiyagarajan · 2018 [cited by examiner]
US 20180330338A1 · Holden et al. · 2018 [cited by applicant]
US 20190244375A1 · Choi et al. · 2019 [cited by applicant]
US 20190261828A1 · Gaus et al. · 2019 [cited by applicant]
US 20190298143A1 · Haidacher et al. · 2019 [cited by applicant]
US 20190365197A1 · Maddux · 2019 [cited by applicant]
US 20200301382A1 · Fawaz et al. · 2020 [cited by applicant]
US 20200397216A1 · Fawaz et al. · 2020 [cited by applicant]
US 20210127939A1 · Hofmann et al. · 2021 [cited by applicant]
US 20210161355A1 · Rahn et al. · 2021 [cited by applicant]
US 20210324561A1 · Kim et al. · 2021 [cited by applicant]
US 20210369076A1 · McGinness et al. · 2021 [cited by applicant]
US 20220095879A1 · Ellingson · 2022 [cited by applicant]
US 20220104680A1 · Han et al. · 2022 [cited by applicant]
CN 101460085A · 2009 [cited by applicant]
CN 101961237A · 2011 [cited by applicant]
CN 107421918A · 2017 [cited by applicant]
CN 107485356A · 2017 [cited by applicant]
CN 107729816A · 2018 [cited by applicant]
CN 104668252B · 2019 [cited by applicant]
CN 109620078A · 2019 [cited by applicant]
CN 110367898A · 2019 [cited by applicant]
CN 111870205A · 2020 [cited by applicant]
DE 102005033345A1 · 2007 [cited by applicant]
DE 102008042290A1 · 2010 [cited by applicant]
DE 102018108775A1 · 2019 [cited by applicant]
EP 0341766A2 · 1989 [cited by applicant]
EP 1272093A2 · 2003 [cited by applicant]
EP 1690924A1 · 2006 [cited by applicant]
EP 1887443A1 · 2008 [cited by applicant]
EP 2497404A1 · 2012 [cited by applicant]
EP 3088593A1 · 2016 [cited by applicant]
JP H02274289A · 1990 [cited by applicant]
JP 05115418A · 1993 [cited by applicant]
JP H0663279 · 1994 [cited by examiner]
JP 2000316783A · 2000 [cited by applicant]
JP 2003038888A · 2003 [cited by applicant]
JP 2004261439A · 2004 [cited by applicant]
JP 2005342143A · 2005 [cited by applicant]
JP 2010036023A · 2010 [cited by applicant]
JP 2015008914A · 2015 [cited by applicant]
JP 2015504708A · 2015 [cited by applicant]
JP 2016193169A · 2016 [cited by applicant]
JP 2018068862A · 2018 [cited by applicant]
KR 102119076B1 · 2020 [cited by applicant]
WO 0178573A2 · 2001 [cited by applicant]
WO 2006002123A1 · 2006 [cited by applicant]
WO 2006097294A1 · 2006 [cited by applicant]
WO 2013090443A1 · 2013 [cited by applicant]
WO 2015080965A1 · 2015 [cited by applicant]
WO 2015167574A1 · 2015 [cited by applicant]
Translation of JPH0663279 by Ioku, published Mar. 8, 1994. [cited by examiner]
Advisory Action from U.S. Appl. No. 17/193,189 dated Aug. 3, 2023, 3 pp. [cited by applicant]
Response to Office Action mailed May 26, 2023, from U.S. Appl. No. 17/193,189, filed Jul. 26, 2023, 14 pp. [cited by applicant]
“NSF/ANSI 3—2017-Commercial Warewashing Equipment,” NSF International, ANSI Standard, Apr. 11, 2017, 42 pp. [cited by applicant]
“Two-Class Logistic Regression,” retrieved from https://docs.microsoft.com/en-us/azure/machine-learning/studio-module-reference/two-class-logistic-regression, May 6, 2019, 7 pp. [cited by applicant]
Brownlee, “A Tour of Machine Learning Algorithms,” machinelearningmastery.com, Aug. 14, 2020, 11 pp. [cited by applicant]
International Preliminary Report on Patentability from International Application No. PCT/US2020/050381, dated Mar. 24, 2022, 11 pp. [cited by applicant]
International Search Report and Written Opinion of International Application No. PCT/US2020/050381, mailed Jan. 15, 2021, 19 pp. [cited by applicant]
Invitation to Pay Additional Fees and, Where Applicable, Protest Fee, Provisional Opinion and Partial Search from International Application No. PCT/US2020/050381, mailed Nov. 16, 2020, 14 pp. [cited by applicant]
Machine Translation of CN 107729816 by Shan et al., published Feb. 23, 2018. [cited by applicant]
Narkhede, “Understanding AUC—ROC Curve,” towardsdatascience.com, Jun. 26, 2018, 7 pp. [cited by applicant]
Narkhede, “Understanding Confusion Matrix,” towardsdatascience.com, May 9, 2018, 6 pp. [cited by applicant]
Patel, “Machine Learning Algorithm Overview,” medium.com, Jul. 21, 2018, 10 pp. [cited by applicant]
Prosecution History from U.S. Appl. No. 17/018,363, dated Dec. 16, 2021 through Mar. 16, 2023, 72 pp. [cited by applicant]
Prosecution History from U.S. Appl. No. 17/193,189, dated May 19, 2022 through May 26, 2023, 88 pp. [cited by applicant]
Prosecution History from U.S. Appl. No. 17/193,507, dated Oct. 21, 2022 through Apr. 28, 2023, 39 pp. [cited by applicant]
Response to Communication Pursuant to Rules 161(1) and 162 EPC dated Apr. 21, 2022, from counterpart European Application No. 20780514.4, filed Oct. 20, 2022, 18 pp. [cited by applicant]
Saslow, “Collinearity—What it Means, Why its Bad, and How Does it Affect Other Models,” medium.com, Jun. 11, 2018, 5 pp. [cited by applicant]
Shung, “Accuracy, Precision, Recall or F1?,” towardsdatascience.com, Mar. 15, 2018, 7 pp. [cited by applicant]
Singh, “Model-Based Feature Importance,” towardsdatascience.com, Jan. 3, 2019, 7 pp. [cited by applicant]
U.S. Appl. No. 18/307,380, filed Apr. 26, 2023, naming inventors Han et al. [cited by applicant]
Youtube, “Regularization Part 1: Ridge (L2) Regression,” retrieved from https://www.youtube.com/watch?app=desktop&v=Q81RR3yKn30&t=3s, Sep. 24, 2018, 1 pp. [cited by applicant]
Youtube, “Regularization Part 2: Lasso (L1) Regression,” Retrieved from https://www.youtube.com/watch?app=desktop&v=NGf0voTMIcs, Oct. 1, 2018, 1 pp. [cited by applicant]
DE102018108775 English translation, accessed on Sep. 2023. (Year: 2019). [cited by applicant]
Notice of Allowance from U.S. Appl. No. 17/193,189 dated Sep. 27, 2023, 14 pp. [cited by applicant]
Office Action from U.S. Appl. No. 17/193,314 dated Sep. 13, 2023, 21 pp. [cited by applicant]
Office Action from counterpart Canadian Application No. 3,152,532 dated Feb. 6, 2024, 4 pp. [cited by applicant]
Response to Office Action mailed Sep. 13, 2023, from U.S. Appl. No. 17/193,314, filed Dec. 13, 2023, 13 pp. [cited by applicant]
U.S. Appl. No. 18/398,859, filed Dec. 28, 2023, by McGinness et al. [cited by applicant]
Communication pursuant to Article 94(3) EPC from counterpart European Application No. 20780514.4 dated Sep. 5, 2024, 5 pp. [cited by applicant]
Office Action, and translation thereof, from counterpart Chinese Application No. 202080058131.9 dated Sep. 2, 2024, 7 pp. [cited by applicant]
Response to Office Action dated Feb. 6, 2024, from counterpart Canadian Application No. 3,152,532 filed Jun. 5, 2024, 31 pp. [cited by applicant]
Final Office Action from U.S. Appl. No. 17/193,314 dated Mar. 27, 2024, 19 pp. [cited by applicant]
Corrected Notice of Allowance from U.S. Appl. No. 18/307,380 dated Dec. 20, 2024, 5 pp. [cited by applicant]
Notice of Allowance from U.S. Appl. No. 18/307,380 dated Dec. 18, 2024, 9 pp. [cited by applicant]
Response to Communication pursuant to Article 94(3) EPC dated Sep. 5, 2024, from counterpart European Application No. 20780514.4 filed Jan. 3, 2025, 18 pp. [cited by applicant]
First Examination Report from counterpart Australian Application No. 2020344611 dated May 19, 2025, 3 pp. [cited by applicant]
Office Action from counterpart Application No. BR 11 2022 002995 0 dated Feb. 25, 2025, 6 pp. Translation Attached. [cited by applicant]
Second Office Action from counterpart Canadian Application No. 3,152,532 dated Mar. 10, 2025, 3 pp. [cited by applicant]
Second Office Action from counterpart Chinese Application No. 202080058131.9 dated May 1, 2025, 15 pp. [cited by applicant]
Response to Response to First Examination Report dated May 19, 2025, from counterpart Australian Application No. 2020344611 filed Aug. 18, 2025, 30 pp. [cited by applicant]
Response to Second Office Action dated Mar. 10, 2025, from counterpart Canadian Application No. 3,152,532 filed Jun. 26, 2025, 23 pp. [cited by applicant]
Third Office Action, and translation thereof, from counterpart Chinese Application No. 202080058131.9 dated Jul. 8, 2025, 16 pp. [cited by applicant]