IP Library › Granted Patent US 12,198,438
Granted Patent B2
US 12,198,438 · App. 18/600,272 · Granted Jan 14, 2025

Systems and methods for monitoring and controlling industrial processes

Inventors: Slawek Frackowiak (Fairfax, VA); Leszek Frackowiak (Vienna, VA)
Assignee: Industrial Video Solutions Inc.
G06V20/52G06T5/50G06T7/0008G06V20/44G06T2207/10048G06T2207/20221G06T2207/30124G06T2207/30136
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,198,438
App. No.
18/600,272
Granted
Jan 14, 2025
Kind
B2
Abstract

Aspects of the present invention provide methods, systems, and/or the like for: (1) receiving first imaging data from a first imaging device, the first imaging data comprising imaging data for at least a first portion of an article of manufacture during a manufacturing process; (2) receiving second imaging data from a second imaging device, the second imaging data comprising imaging data for at least the first portion of an article of manufacture during a manufacturing process; (3) receiving event data associated with a manufacturing event associated with the article of manufacture during the manufacturing process; (4) processing, to produce a first data analysis result, the first imaging data, the second imaging data, and the event data using a machine-learning model trained with historical imaging data and historical manufacturing event data; (5) determining based on the first data analysis result, an origin of the manufacturing event.

Claims (51)

1. A method comprising:

receiving, by computing hardware, first imaging data from a first imaging device, the first imaging data comprising imaging data for at least a first portion of an article of manufacture during a manufacturing process;

receiving, by the computing hardware, second imaging data from a second imaging device, the second imaging data comprising imaging data for at least the first portion of an article of manufacture during a manufacturing process;

receiving, by the computing hardware, event data associated with a manufacturing event associated with the article of manufacture during the manufacturing process;

processing, by the computing hardware to produce a first data analysis result, the first imaging data, the second imaging data, and the event data using a machine-learning model trained with historical imaging data and historical manufacturing event data;

determining, by the computing hardware based on the first data analysis result, an origin of the manufacturing event; and

generating, by the computing hardware, a user interface for display on a computing device, the user interface comprising an indication of the origin of the manufacturing event, wherein generating the user interface comprises:

merging the first imaging data and the second imaging data such that the user interface comprises a display of the first portion of the article of manufacture from the first imaging data and the second imaging data; and

modifying the display of the first portion of the article of manufacture from the first imaging data and the second imaging data to include the indication of the origin of the manufacturing event.

2. The method of claim 1 , wherein:

determining the origin of the manufacturing event comprises determining a location of the origin of the manufacturing event on the first portion of the article of manufacture; and

the indication of the origin of the manufacturing event comprises the location.

3. The method of claim 1 , wherein the first imaging data comprises imaging data of a first type and the second imaging data comprises imaging data of a second type.

4. The method of claim 1 , wherein the first imaging data comprises infrared imaging data and the second imaging data comprises visual imaging data.

5. The method of claim 1 , wherein the manufacturing event comprises at least one of: a paper breakage event, identification of a portion of the article of manufacture having a property that deviates from a desired property, or identification of an issue with the manufacturing process.

6. The method of claim 1 , wherein:

the article of manufacture comprises at least one of a paper web, a non-paper web, an extruded material, a polymer, or a metal; and

the manufacturing event comprises at least one of a breakage event, an identification of a defect, or an identification of a manufacturing process issue.

7. A computer-implemented data processing method for identifying a cause of a manufacturing process event, the method comprising:

receiving, by computing hardware, first imaging data of a first type from one or more imaging devices, the first imaging data comprising imaging data for at least a first portion of a paper web during a paper manufacturing process;

receiving, by the computing hardware, second imaging data of a second type from one or more imaging devices, the second imaging data comprising imaging data for at least a second portion of a paper web during a paper manufacturing process;

receiving, by the computing hardware, event data associated with a manufacturing issue experienced during the paper manufacturing process;

processing, by the computing hardware, the first imaging data, the second imaging data, and the event data using at least one of a machine-learning model trained or a rules-based model to generate a prediction as to a cause of the manufacturing event;

determining, by the computing hardware based on the prediction, the cause of the manufacturing event; and

providing, by the computing hardware, an indication of the cause of the manufacturing event for display on a computing device;

synchronizing, by the computing hardware, the first imaging data and the second imaging data to generate third imaging data comprising the first imaging data and the second imaging data;

modifying the third imaging data to include the indication of the cause of the manufacturing event; and

providing the third imaging data for display on the computing device.

8. The method of claim 7 , further comprising:

receiving, via the computing device, user input related to the cause of the manufacturing event; and

providing the user input to at least one of the machine-learning model trained, the classification model, or the rules-based model as training data for generating a prediction as to cause a future manufacturing event similar to the manufacturing event.

9. The method of claim 7 , wherein:

the article of manufacture comprises at least one of a paper web, a non-paper web, an extruded material, a polymer, or a metal; and

the manufacturing event comprises at least one of a breakage event, an identification of a defect, or an identification of a manufacturing process issue.

10. The method of claim 7 , wherein processing the first imaging data, the second imaging data, and the event data using at least one of a machine-learning model trained or a rules-based model to generate a prediction as to a cause of the manufacturing event occurs in response to an identification of the manufacturing event.

11. The method of claim 7 , wherein the first portion of the paper web is the second portion of the paper web.

12. A method comprising:

receiving, by computing hardware, first imaging data from a first imaging device, the first imaging data comprising imaging data for at least a first portion of an article of manufacture during a manufacturing process;

receiving, by the computing hardware, second imaging data from a second imaging device, the second imaging data comprising imaging data for at least the first portion of an article of manufacture during a manufacturing process;

receiving, by the computing hardware, event data associated with a manufacturing event associated with the article of manufacture during the manufacturing process;

processing, by the computing hardware to produce a first data analysis result, the first imaging data, the second imaging data, and the event data using a machine-learning model trained with historical imaging data and historical manufacturing event data;

determining, by the computing hardware based on the first data analysis result, an origin of the manufacturing event; and

generating, by the computing hardware, a user interface for display on a computing device, the user interface comprising an indication of the origin of the manufacturing event, wherein:

determining the origin of the manufacturing event comprises determining a location of the origin of the manufacturing event on the first portion of the article of manufacture; and

the indication of the origin of the manufacturing event comprises the location.

13. The method of claim 12 , wherein the first imaging data comprises imaging data of a first type and the second imaging data comprises imaging data of a second type.

14. The method of claim 12 , wherein the first imaging data comprises infrared imaging data and the second imaging data comprises visual imaging data.

15. The method of claim 12 , wherein the manufacturing event comprises at least one of: a paper breakage event, identification of a portion of the article of manufacture having a property that deviates from a desired property, or identification of an issue with the manufacturing process.

16. The method of claim 12 , wherein:

the article of manufacture comprises at least one of a paper web, a non-paper web, an extruded material, a polymer, or a metal; and

the manufacturing event comprises at least one of a breakage event, an identification of a defect, or an identification of a manufacturing process issue.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 13, 2024
From: FRACKOWIAK, SLAWEK; FRACKOWIAK, LESZEK
To: INDUSTRIAL VIDEO SOLUTIONS INC.
Reel/Frame 066750/0514 →
Continuity (6)
Continuation In Part 18431571 · Feb 2, 2024
Continuation In Part 18212548 · Jun 21, 2023
Continuation In Part 18131926 · Apr 7, 2023
Provisional Application 63470057 · May 31, 2023
Provisional Application 63394805 · Aug 3, 2022
Related Publication 20240212356A1 · Jun 27, 2024
References Cited (125)
US 1338094A · Pope · 1920 [cited by examiner]
US 4988875A · Ortiz · 1991 [cited by examiner]
US 5006722A · Adelson · 1991 [cited by applicant]
US 5233415A · French et al. · 1993 [cited by applicant]
US 5389789A · Nguyen · 1995 [cited by applicant]
US 5862608A · Kotitschke · 1999 [cited by applicant]
US 5942689A · Bonissone et al. · 1999 [cited by applicant]
US 5999636A · Juang · 1999 [cited by applicant]
US 6075590A · Edgar · 2000 [cited by examiner]
US 6135000A · Caspar et al. · 2000 [cited by applicant]
US 6158287A · Satake et al. · 2000 [cited by applicant]
US 6362889B1 · Mustonen · 2002 [cited by applicant]
US 6388749B1 · Yamashita et al. · 2002 [cited by applicant]
US 6410916B1 · Jost et al. · 2002 [cited by applicant]
US 6463170B1 · Toivonen et al. · 2002 [cited by applicant]
US 6466877B1 · Chen et al. · 2002 [cited by applicant]
US 6498993B1 · Chen et al. · 2002 [cited by applicant]
US 6519534B2 · Chen et al. · 2003 [cited by applicant]
US 6522978B1 · Chen et al. · 2003 [cited by applicant]
US 6542852B2 · Chen et al. · 2003 [cited by applicant]
US 6615511B2 · Augscheller et al. · 2003 [cited by applicant]
US 6849851B2 · Komulainen et al. · 2005 [cited by applicant]
US 6943913B1 · Jung · 2005 [cited by applicant]
US 7138036B2 · Yamashita et al. · 2006 [cited by applicant]
US 7494567B2 · Haran · 2009 [cited by applicant]
US 7688079B2 · Shioda et al. · 2010 [cited by applicant]
US 7750330B2 · Murata · 2010 [cited by applicant]
US 7934722B2 · Namikawa · 2011 [cited by applicant]
US 8023843B2 · Degruchy et al. · 2011 [cited by applicant]
US 8265498B2 · Fan et al. · 2012 [cited by applicant]
US 8396384B2 · Hayashihara et al. · 2013 [cited by applicant]
US 8477234B2 · Tanigawa et al. · 2013 [cited by applicant]
US 8494384B2 · Furuya et al. · 2013 [cited by applicant]
US 8532949B2 · Teh et al. · 2013 [cited by applicant]
US 9401015B2 · Minekawa · 2016 [cited by examiner]
US 9838635B2 · Gousev et al. · 2017 [cited by applicant]
US 10552662B2 · Chang et al. · 2020 [cited by applicant]
US 10875024B2 · Li et al. · 2020 [cited by applicant]
US 10884401B2 · Valkonen · 2021 [cited by applicant]
US 10946675B2 · Takeishi · 2021 [cited by applicant]
US 11142422B2 · Paanasalo · 2021 [cited by applicant]
US 11231363B2 · Shitara · 2022 [cited by applicant]
US 11297388B2 · Menendez · 2022 [cited by applicant]
US 11816893B1 · Frackowiak · 2023 [cited by examiner]
US 11846930B1 · Frackowiak · 2023 [cited by examiner]
US 11932991B2 · Frackowiak · 2024 [cited by examiner]
US 20020023369A1 · Augscheller · 2002 [cited by examiner]
US 20020052699A1 · Chen · 2002 [cited by examiner]
US 20020066545A1 · Karjanmaa · 2002 [cited by applicant]
US 20020166970A1 · Komulainen · 2002 [cited by examiner]
US 20040112560A1 · Yamashita · 2004 [cited by examiner]
US 20040218795A1 · Zhou · 2004 [cited by examiner]
US 20050079386A1 · Brown et al. · 2005 [cited by applicant]
US 20050094160A1 · Murai et al. · 2005 [cited by applicant]
US 20050167620A1 · Cho · 2005 [cited by examiner]
US 20050169509A1 · Grasslin et al. · 2005 [cited by applicant]
US 20060096726A1 · Ahvenainen et al. · 2006 [cited by applicant]
US 20060102839A1 · Bhaskar · 2006 [cited by examiner]
US 20060143671A1 · Ens et al. · 2006 [cited by applicant]
US 20060196621A1 · Johansson · 2006 [cited by applicant]
US 20060258246A1 · Bomberger et al. · 2006 [cited by applicant]
US 20070058089A1 · Wang · 2007 [cited by applicant]
US 20070137823A1 · Haran · 2007 [cited by examiner]
US 20070200568A1 · Shioda · 2007 [cited by examiner]
US 20070294284A1 · Evans · 2007 [cited by examiner]
US 20080311416A1 · Kelly et al. · 2008 [cited by applicant]
US 20090028416A1 · Floeder · 2009 [cited by examiner]
US 20090028417A1 · Floeder · 2009 [cited by examiner]
US 20090030544A1 · Floeder · 2009 [cited by examiner]
US 20130120561A1 · Heintze · 2013 [cited by examiner]
US 20130255096A1 · Kampris et al. · 2013 [cited by applicant]
US 20140098360A1 · Kwon · 2014 [cited by applicant]
US 20140286563A1 · Frackowiak · 2014 [cited by examiner]
US 20150013556A1 · Slaats · 2015 [cited by applicant]
US 20160267111A1 · Shoaib et al. · 2016 [cited by applicant]
US 20170015832A1 · Berlin et al. · 2017 [cited by applicant]
US 20180347970A1 · Sasaki · 2018 [cited by examiner]
US 20180348146A1 · Ando · 2018 [cited by examiner]
US 20190129396A1 · Valkonen · 2019 [cited by examiner]
US 20190139212A1 · Hanzawa · 2019 [cited by examiner]
US 20200130976A1 · Paanasalo · 2020 [cited by examiner]
US 20200184630A1 · Wang · 2020 [cited by examiner]
US 20210010954A1 · Adler · 2021 [cited by examiner]
US 20210011177A1 · Adler · 2021 [cited by examiner]
US 20210025687A1 · Ichikawa · 2021 [cited by applicant]
US 20210209745A1 · Jiang et al. · 2021 [cited by applicant]
US 20210287354A1 · Kumar · 2021 [cited by examiner]
US 20210380364A1 · Gumpinger et al. · 2021 [cited by applicant]
US 20220020139A1 · Weiss et al. · 2022 [cited by applicant]
US 20220076411A1 · Georgescu et al. · 2022 [cited by applicant]
US 20220236176A1 · Bolton et al. · 2022 [cited by applicant]
US 20220270229A1 · Nokelby · 2022 [cited by applicant]
US 20230074247A1 · Xia · 2023 [cited by examiner]
US 20230137432A1 · Selim · 2023 [cited by examiner]
US 20230147668A1 · Penugonda · 2023 [cited by applicant]
US 20230169642A1 · Floeder · 2023 [cited by examiner]
US 20230237640A1 · Ekawa · 2023 [cited by examiner]
US 20230260413A1 · Lee · 2023 [cited by examiner]
US 20230286261A1 · Condello et al. · 2023 [cited by applicant]
US 20230342907A1 · Katsuyama · 2023 [cited by examiner]
US 20230401589A1 · Panchamgam · 2023 [cited by examiner]
US 20240044079A1 · Frackowiak · 2024 [cited by examiner]
US 20240046443A1 · Okazaki · 2024 [cited by examiner]
US 20240046617A1 · Adolf · 2024 [cited by examiner]
US 20240071059A1 · Schreiner · 2024 [cited by examiner]
US 20240094139A1 · Smith · 2024 [cited by examiner]
JP 08022320 · 1996 [cited by applicant]
JP 2006099497 · 2006 [cited by applicant]
KR 1020210002142 · 2021 [cited by applicant]
KR 1020210150834 · 2021 [cited by applicant]
WO 2017037896 · 2017 [cited by applicant]
WO 2022043024 · 2022 [cited by applicant]
International Search Report, dated Nov. 21, 2023, from corresponding International Application No. PCT/US2023/029385. [cited by applicant]
International Search Report, dated Nov. 23, 2023, from corresponding International Application No. PCT/US2023/029389. [cited by applicant]
Notice of Allowance, dated Aug. 22, 2023, from corresponding U.S. Appl. No. 18/131,920. [cited by applicant]
Notice of Allowance, dated Nov. 24, 2023, from corresponding U.S. Appl. No. 18/212,548. [cited by applicant]
Notice of Allowance, dated Oct. 25, 2023, from corresponding U.S. Appl. No. 18/131,926. [cited by applicant]
Office Action, dated Jul. 13, 2023, from corresponding U.S. Appl. No. 18/131,926. [cited by applicant]
Office Action, dated Sep. 20, 2023, from corresponding U.S. Appl. No. 18/212,548. [cited by applicant]
Written Opinion of the International Searching Authority, dated Nov. 21, 2023, from corresponding International Application No. PCT/US2023/029385. [cited by applicant]
Written Opinion of the International Searching Authority, dated Nov. 23, 2023, from corresponding International Application No. PCT/US2023/029389. [cited by applicant]
Office Action, dated Apr. 22, 2024, from corresponding U.S. Appl. No. 18/431,571. [cited by applicant]
Office Action, dated Jun. 12, 2024, from corresponding U.S. Appl. No. 18/600,258. [cited by applicant]
Notice of Allowance, dated Jun. 26, 2024, from corresponding U.S. Appl. No. 18/431,571. [cited by applicant]
Notice of Allowance, dated Oct. 11, 2024, from corresponding U.S. Appl. No. 18/767,569. [cited by applicant]