IP Library Granted Patent US 10,824,126
Granted Patent B2
US 10,824,126 · App. 14/054,531 · Granted Nov 3, 2020

Device and method for the gesture control of a screen in a control room

Inventors: Jens Doppelhamer (Ladenburg, DE); Kristoffer Husoy (Oslo, NO); Martin Hollender (Dossenheim, DE); Mikko Rissanen (Kungsängen, SE); Torgeir Enkerud (Oslo, NO)
Assignee: ABB Schweiz AG
G05B15/02G05B19/0423G05B2219/35444
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,824,126
App. No.
14/054,531
Granted
Nov 3, 2020
Kind
B2
Abstract

An operation and observation system of a technical plant and/or a technical process and associated method are provided for operating components and for displaying measurement readings, process variables and/or status messages of the components of the plant and/or process. The system includes at least one large screen for displaying the components of the technical process and/or technical plant, and a plurality of operator workstations. A movement detection and movement control component is installed on the screen and on the operator workstations, which allows an operation and/or polling of the components, the measurement readings, the process variables and/or the status messages of the components of the technical plant or of the technical process by means of a body movement of an operator.

Claims (53)

1. An operating and observation system for a technical installation, the operating and observation system comprising:

a gesture controller configured to operate components of the technical installation and for presenting at least one of measured values, process variables and state messages from the components of the technical installation in a control room;

at least one large display screen in the control room, and a plurality of user workstations for presenting, in the control room, the components of the technical installation and the at least one of the measured values, process variables and state messages from the components of the technical installation; and

a plurality of sensors each configured to recognize gestures constituting physical movements,

wherein the large display screen and the user workstations respectively have therein installed a motion recognition and motion controller configured to recognize and control motion,

wherein the motion recognition and motion controller is configured to allow at least one of operation and polling of at least one of the components, the measured values, the process variables and the state messages from the components of the at least one of the technical installation by means of a gesture from at least one user in the control room, the gesture constituting a physical movement,

wherein the motion recognition and motion controller comprises a position recognition controller configured to determine a position of individual users of the technical installation within the control room, by using a model of the control room to identify a target system in the control room to which a gesture of the at least one user relates, the model using a first component to depict the control room and showing where at least one device is situated in the control room, and

wherein the motion recognition and motion controller is configured to, based on the gesture of the at least one user, send selection and position information to the identified target system for evaluation and initiation of appropriate actions,

wherein the motion recognition and motion controller is further configured to:

provide an identification for a person who is in front of the large screen and an identification of the person is usable as a basis for presenting views tailored specifically to the identified person.

2. The operating and observation system as claimed in claim 1 , wherein the views that have sensitive information are automatically closed if there is no identification for a relevant person.

3. The operating and observation system as claimed in claim 1 , comprising:

at least one recording unit configured to record spoken words and interact with a processing unit connected thereto that is configured to produce from the recorded spoken words an execution signal to execute at least one of the operation and polling of the components, the measured values, the process variables, and the state messages from the components of the technical installation.

4. The operating and observation system as claimed in claim 3 , wherein the large screen and the user workstations comprise a respective processor configured to execute voice recognition software tangibly recorded on a non-transitory computer-readable recording medium to execute the at least one recording unit.

5. The operating and observation system as claimed in claim 4 , wherein at least one of selected components, measured values, process variables and state messages from the components are presented as objects to be operated by spoken commands.

6. The operating and observation system as claimed in claim 1 , comprising:

at least one recording unit configured to record previously defined movements and gestures from a user, convert the previously defined recorded movements and gestures into a motion signal, and provide the motion signal for the motion recognition and motion controller installed on the large screen and the user workstations for further processing.

7. The operating and observation system as claimed in claim 6 , wherein the motion recognition and motion controller is further configured to:

provide an identification for a person who is in front of the large screen and an identification of the person is usable as a basis for presenting views tailored specifically to the identified person.

8. The operating and observation system as claimed in claim 7 , wherein the views that have sensitive information are automatically closed if there is no identification for a relevant person.

9. The operating and observation system as claimed in claim 7 , comprising:

at least one recording unit configured to record spoken words and interact with a processing unit connected thereto that is configured to produce from the recorded spoken words an execution signal to execute at least one of the operation and polling of the components, the measured values, the process variables, and the state messages from the components of the installation.

10. The operating and observation system as claimed in claim 9 , wherein the large screen and the user workstations comprise a respective processor configured to execute voice recognition software tangibly recorded on a non-transitory computer-readable medium to execute the at least one recording unit.

11. The operating and observation system as claimed in claim 10 , wherein at least one of selected components, measured values, process variables and state messages from the components are presented as objects to be operated by spoken commands.

12. The operating and observation system as claimed in claim 1 , wherein the motion recognition and motion controller is configured to:

detect a gesture to move a first representation of a telephone in the control room to a second representation of a loudspeaker; and

operate, in response to the detection of the gesture, the telephone in a conference mode.

13. The operating and observation system as claimed in claim 1 , wherein the motion recognition and motion controller is configured to:

detect a gesture indicative of a gripping movement; and

control, in response to the detection of the gesture, an operation of at least one of a motor, a pump, or a valve as a function of the gripping movement.

14. The operating and observation system as claimed in claim 1 , wherein the motion recognition and motion controller is configured to:

store a hierarchical structure of data indicative of components of the technical installation; and

present information associated with a first level of the hierarchical structure;

detect one or more gestures indicative of a request to navigate to a different level of the hierarchical structure; and

present, in response to the detection of the one or more gestures, information associated with the different level of the hierarchical structure.

15. The operating and observation system as claimed in claim 1 , wherein the identified target system is one of the user workstations, and wherein to send the selection and position information further comprises to send a request to the user workstation to transfer information previously controlled by the user workstation to the large display screen.

16. The operating and observation system as claimed in claim 1 , wherein the motion recognition and motion controller is configured to:

determine a distance of each user from the large display screen;

identify the distance of the user situated closest to the large display screen;

determine, as function of the identified distance of the user situated closest to the large display screen, a level of detail of information to be presented on the large display screen.

17. The operating and observation system as claimed in claim 16 , wherein the motion recognition and motion controller is further configured to select, as a function of the identified distance of the user situated closest to the large display screen, a font size to be used for presentation of the information on the large display screen.

18. The operating and observation system as claimed in claim 1 , wherein the motion recognition and motion controller is further configured to:

determine an amount of time that has elapsed since a gesture has been recognized;

determine whether the elapsed amount of time satisfies a predefined threshold; and

activate, in response to a determination that the elapsed amount of time satisfies the predefined threshold, an audible signal in the control room.

19. A method for operating components of a technical installation with a gesture controller and for presenting at least one of measured values, process variables, and state messages from the components of the technical installation in a control room, the method comprising:

presenting, on at least one of a large display screen and a multiplicity of user workstations in the control room, the components, measured values, process variables and state messages from the components of the technical installation;

recognizing gestures by at least one of a plurality of sensors, the gestures constituting physical movements; and

operating, on at least one of the large display screen and the user workstations, a respective motion recognition and motion controller to execute at least one of operation and polling of at least one of the components, the measured values, the process variable and the state message from the components of the at least one of the technical installation by means of a gesture from at least one user in the control room, the gesture constituting a physical movement,

wherein the operation of the motion recognition and motion controller comprises operating a position recognition controller to determine a position of individual users of the technical installation within the control room, by using a model of the control room to identify a target system in the control room to which a gesture of the at least one user relates, the model using a first component to depict the control room and showing where at least one device is situated in the control room, and

wherein the motion recognition and motion controller, based on the gesture of the at least one user, sends selection and position information to the identified target system for evaluation and initiation of appropriate actions,

wherein the motion recognition and motion controller is further configured to:

provide an identification for a person who is in front of the large screen and an identification of the person is usable as a basis for presenting views tailored specifically to the identified person.

Assignments (3)
MERGER Recorded Sep 24, 2020
From: ABB TECHNOLOGY AG
To: ABB SCHWEIZ AG
Reel/Frame 054188/0770 →
MERGER Recorded Nov 15, 2016
From: ABB TECHNOLOGY LTD.
To: ABB SCHWEIZ AG
Reel/Frame 040622/0128 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 14, 2014
From: DOPPELHAMER, JENS; HUSOY, KRISTOFFER; HOLLENDER, MARTIN; RISSANEN, MIKKO; ENKERUD, TORGEIR
To: ABB TECHNOLOGY AG
Reel/Frame 031964/0321 →