IP Library › Granted Patent US 12,282,320
Granted Patent B2
US 12,282,320 · App. 17/843,122 · Granted Apr 22, 2025

Adding model state to human machine interface (HMI) views

Inventors: Jordan C. Reynolds (Austin, TX); John J. Hagerbaumer (Mequon, WI); Troy W. Mahr (Pleasant Prairie, WI); Thomas K. Jacobsen (Wake Forest, NC); Giancarlo Scaturchio (Pisa, IT)
Assignee: Rockwell Automation Technologies, Inc.
G05B19/41885G05B19/0426G05B2219/13144
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,282,320
App. No.
17/843,122
Granted
Apr 22, 2025
Kind
B2
Abstract

Various embodiments of the present technology generally relate to industrial automation environments. More specifically, embodiments include systems and methods to visualize machine learning model status in an industrial automation environment. In some examples, a machine learning component receives process inputs associated with industrial devices in the industrial automated environment. The machine learning component processes the inputs to generate machine learning outputs and transfers the machine learning outputs to influence one or more functions of the industrial devices. The machine learning component reports operational data characterizing the machine learning outputs. A Human Machine Interface (HMI) component displays a visualization of the machine learning component and receives the operational data from the machine learning component. In response to a user selection, the HMI component displays an expanded view of the machine learning component that comprises the operational data and that indicates relationships between the machine learning component and the industrial devices.

Claims (50)

1. A system to visualize machine learning model status in an industrial automation environment, the system comprising:

a memory that stores executable components; and

a processor, operatively coupled to the memory, that executes the executable components, the executable components comprising:

a machine learning component configured to receive process inputs associated with industrial devices in the industrial automated environment, process the inputs to generate machine learning outputs that comprise control signaling to influence one or more functions of the industrial devices, transfer the control signaling to a Programmable Logic Controller (PLC), and report operational data characterizing the machine learning outputs and the control signaling, wherein the PLC influences one or more functions of the industrial devices based on the control signaling;

a Human Machine Interface (HMI) component hosted by an on-site HMI device configured to display a visualization of the machine learning component and receive the operational data from the machine learning component; and

in response to a user selection, the HMI component configured to display an expanded view of the machine learning component and that comprises the operational data that characterizes the machine learning outputs and the control signaling and that indicates operational relationships between the machine learning component, the PLC, and the industrial devices.

2. The system of claim 1 wherein:

the HMI component is configured to display a list of the machine learning outputs and information context associated with individual ones of the machine learning outputs.

3. The system of claim 2 wherein the information context indicates ones of the industrial devices influenced by the control signaling.

4. The system of claim 1 wherein:

the machine learning outputs comprises machine learning predictions; and

the HMI component is configured to display a predication accuracy percentage for the machine learning component based on the machine learning predictions.

5. The system of claim 4 wherein the predication accuracy percentage for the machine learning model comprises an r-squared value.

6. The system of claim 1 wherein:

the machine learning outputs comprises machine learning predictions; and

the HMI component is configured to display a percent accuracy for the machine learning component based on the machine learning predictions; and

the HMI component is further configured to receive user input to disable the machine learning component and transfer signaling to an industrial control component to disable the machine learning component; and further comprising:

the industrial control component configured to receive the signaling and responsively disable the machine learning component.

7. The system of claim 1 wherein the HMI component is configured to display control signaling context for the machine learning component.

8. A method to visualize machine learning model status in an industrial automation environment, the method comprising:

a machine learning component receiving process inputs associated with industrial devices in the industrial automated environment, processing the inputs to generate machine learning outputs that comprise control signaling to influence one or more functions of the industrial devices, transferring the control signaling to a Programmable Logic Controller (PLC), and reporting operational data characterizing the machine learning outputs and control signaling, wherein the PLC influences one or more functions of the industrial devices based on the control signaling;

a Human Machine Interface (HMI) component hosted by an on-site HMI device displaying a visualization of the machine learning component and receiving the operational data from the machine learning component; and

in response to a user selection, the HMI component displaying an expanded view of the machine learning component and that comprises the operational data that characterizes the machine learning outputs and the control signaling and that indicates operational relationships between the machine learning component, the PLC, and the industrial devices.

9. The method of claim 8 wherein:

the HMI component displaying the expanded view comprises the HMI component displaying a list of the machine learning outputs and information context associated with individual ones of the machine learning outputs.

10. The method of claim 9 wherein the information context indicates ones of the industrial devices influenced by the control signaling.

11. The method of claim 8 wherein:

the machine learning outputs comprises machine learning predictions; and

the HMI component displaying the expanded view comprises the HMI component displaying a predication accuracy percentage for the machine learning component based on the machine learning predictions.

12. The method of claim 11 wherein the predication accuracy percentage for the machine learning model comprises an r-squared value.

13. The method of claim 8 wherein:

the machine learning outputs comprises machine learning predictions; and

the HMI component displaying the expanded view comprises the HMI component displaying a percent accuracy for the machine learning component based on the machine learning predictions; and further comprising:

the HMI component receiving user input to disable the machine learning component and transferring signaling to an industrial control component to disable the machine learning component; and

the industrial control component receiving the signaling and responsively disabling the machine learning component.

14. The method of claim 8 wherein the HMI component displaying the expanded view comprises the HMI component displaying control signaling context for the machine learning component.

15. A non-transitory computer-readable medium storing instructions to visualize machine learning model status in an industrial automation environment, wherein the instructions, in response to execution by a processor, cause the processor to drive a system to perform operations comprising:

receiving process inputs associated with industrial devices in the industrial automated environment, processing the inputs to generate machine learning outputs that comprise control signaling to influence one or more functions of the industrial devices, and transferring the control signaling to a Programmable Logic Controller (PLC) wherein the PLC influences one or more functions of the industrial devices based on the control signaling;

displaying, via an on-site Human Machine Interface (HMI), a visualization of a machine learning component; and

in response to a user selection via the on-site HMI, displaying an expanded view of the machine learning component that comprises operational data that characterizes the machine learning outputs and the control signaling and operational relationships between the machine learning component, the PLC, and the industrial devices.

16. The non-transitory computer-readable medium of claim 15 , the operations further comprising:

displaying a list of the machine learning outputs and information context associated with individual ones of the machine learning outputs.

17. The non-transitory computer-readable medium of claim 16 wherein the information context indicates ones of the industrial devices influenced by the control signaling.

18. The non-transitory computer-readable medium of claim 15 , the operations further comprising:

displaying a predication accuracy percentage for the machine learning component based on machine learning predictions.

19. The non-transitory computer-readable medium of claim 18 wherein the predication accuracy percentage for the machine learning model comprises an r-squared value.

20. The non-transitory computer-readable medium of claim 15 , the operations further comprising:

displaying a percent accuracy for the machine learning component based on machine learning predictions;

receiving user input to disable the machine learning component; and

transferring signaling to an industrial control component to disable the machine learning component.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 20, 2022
From: HAGERBAUMER, JOHN H.
To: ROCKWELL AUTOMATION TECHNOLOGIES, INC.
Reel/Frame 061152/0853 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 17, 2022
From: REYNOLDS, JORDAN C.; MAHR, TROY W.; JACOBSEN, THOMAS K.; SCATURCHIO, GIANCARLO
To: ROCKWELL AUTOMATION TECHNOLOGIES, INC.
Reel/Frame 060237/0169 →
Continuity (1)
Related Publication 20230409021A1 · Dec 21, 2023
References Cited (17)
US 10216177B2 · Gildert et al. · 2019 [cited by applicant]
US 11740624B2 · Ramamurthy et al. · 2023 [cited by applicant]
US 20120158623A1 · Bilenko · 2012 [cited by examiner]
US 20190102657A1 · Sayyar · 2019 [cited by examiner]
US 20190354093A1 · Nguyen et al. · 2019 [cited by applicant]
US 20200103838A1 · Bertinetti et al. · 2020 [cited by applicant]
US 20200150638A1 · Mourzine · 2020 [cited by examiner]
US 20210090105A1 · Nordmark et al. · 2021 [cited by applicant]
US 20210097456A1 · Sayyarrodsari · 2021 [cited by examiner]
US 20210389748A1 · McAdam · 2021 [cited by examiner]
US 20220292338A1 · Nair · 2022 [cited by examiner]
US 20220388160A1 · Yonemoto et al. · 2022 [cited by applicant]
US 20220390515A1 · Pickerd · 2022 [cited by examiner]
US 20230145448A1 · Huber et al. · 2023 [cited by applicant]
Notice of Allowance in U.S. Appl. No. 17/826,689, mailed Aug. 6, 2024, 19 pages. [cited by applicant]
Non-Final Office Action issued by the United States Patent and Trademark Office in U.S. Appl. No. 17/851,628 on Sep. 13, 2024, 24 pages. [cited by applicant]
Notice of Allowance and Fee(s) Due issued by the United States Patent and Trademark Office in U.S. Appl. No. 17/826,689 on Aug. 6, 2024, 18 pages. [cited by applicant]