Method and device for hierarchically planning executions of operation sequences achieving goal states of technical system workflows and checking them through system simulation or system hardware execution

1 . A method for hierarchically planning executions of operation sequences achieving goal states of technical system workflows and checking the executions of operation sequences achieving goal states through hardware execution, wherein the method comprises:

a) for a technical system workflow of a technical system, wherein the technical system is a robotic system having robotic arms and is configured to perform physical actions, a workflow related planning domain is given by deploying graphical-user-interface-based inputs comprising workflow related facts, wherein the workflow related facts are based on both a digital twin and a workflow related set of planning operators, and wherein the digital twin is a 3 dimensional (3D) representation of the technical system,

b) specifying a workflow related goal state by the graphical-user-interface-based inputs of a goal-state-specifying-process given by consecutive steps,

b1) a first step of receiving a choice from a user using a 3D editor, each time in doing so respectively of one different initial goal state, wherein the one different initial goal state is one of the graphical-user-interface-based inputs,

b2) a second step of generating a list depending on a chosen initial goal state number “mGsi” with mesi being chosen from a number of different goal state abstraction levels, and

b3) a third step of receiving a user selection from the generated list of one different goal state abstraction level of the mesi different goal state abstraction levels, wherein the selected one different goal state abstraction level is one of the graphical-user-interface-based inputs,

c) defining a planning task, which is a combination of facts, for the specified workflow related goal state and according to the workflow related planning domain based on the facts and for at least one selected planning operator of the workflow related set of planning operators, wherein each planning operator of the at least one selected planning operator is a task capturing physical actions of the technical system workflow, and

d) by a planning approach according to the defined planning task

d1) checking executions of the at least one selected planning operator iteratively in order to find a full plan comprising first operation sequences to achieve a goal state of the goal states by a primary query loop, wherein in consecutive loop iterations until, by <i> executing in hardware the at least one selected planning operator, <ii> generating primary control commands due to the at least one executed selected planning operator, <iii> accordingly applying the primary control commands to simulate the technical system workflow and <iv> accordingly checking system simulation through primary measurement results of measurements, the full plan achieving the goal state is found, and

d2) checking executions of the full plan achieving the goal state iteratively by a secondary query loop, wherein in consecutive loop iterations until by <i> executing the full plan, <ii> generating secondary control commands due to the full plan, <iii> performing, by applying the secondary control commands, a hardware execution of the technical system workflow and <iv> accordingly checking the hardware execution of the technical system workflow through secondary measurement results of measurements, the goal state is achieved.

2 . The method according to claim 1 , wherein the workflow related goal state specified by the graphical-user-interface-based inputs of the goal-state-specifying-process is given after the third step by further consecutive steps,

b4) a fourth step of sampling different goal states of selected different goal state abstraction levels and presenting the sampled different goal states to the user as presented goal states, and

b5) a fifth step of interrogating the user whether all the presented goal states are feasible, and if all the presented goal states are feasible then the different initial goal state and the selected different goal state abstraction levels are both the graphical-user-interface-based inputs; otherwise if all the presented goal states are not feasible either (“Option I”) the third step is carried out for another selected different goal state abstraction level or (“Option II”) the first step is carried out for another chosen different initial goal state.

3 . The method according to claim 1 , wherein one or more planning operators of the at least one selected planning operator are selectable for checking the executions of the at least one selected planning operator iteratively in order to find the full plan comprising the first operation sequences.

4 . The method according to claim 1 , said method further comprising outputting the technical system workflow via a data logger of the technical system.

5 . The method according to claim 1 , wherein the physical actions of the robotic system include one or more physical actions selected from the group consisting of torques, motions, and graspings.

6 . A computer-implemented tool, a computer-program-product, configured as an application software, called as APP, for hierarchically planning executions of operation sequences achieving goal states of technical system workflows and checking the executions of operation sequences achieving goal states through hardware execution with

a non-transitory, processor-readable storage medium having processor-readable program-instructions of a program module for planning and checking the executions of the operation sequences, which is stored in the non-transitory, processor-readable storage medium, and

a processor connected with the non-transitory, processor-readable storage medium executing the processor-readable program-instructions of the program module to carry out the method according to claim 1 .

7 . A device for hierarchically planning executions of operation sequences achieving goal states of technical system workflows and checking the executions of operation sequences achieving goal states through system simulation or hardware execution, with a processor, a graphical user interface and a communications interface each connected to the processor and a non-transitory, processor-readable storage medium having processor-readable program-instructions of a program module for planning and checking the executions of the operation sequences, which is stored in the non-transitory, processor-readable storage medium, wherein the processor executing the processor-readable program-instructions of the program module, the graphical user interface, the communications interface and the non-transitory, processor-readable storage medium with the processor-readable program-instructions of the program module form a functional unit which is configured to perform a method that comprises:

a) for a technical system workflow of a technical system wherein the technical system is a robotic system having robotic arms and is configured to perform physical actions, a workflow related planning domain is given by deploying graphical-user-interface-based inputs comprising workflow related facts, wherein the workflow related facts are based on both a digital twin and a workflow related set of planning operators, and wherein the digital twin is a 3 dimensional (3D) representation of the technical system,

b) specifying a workflow related goal state inputted via the graphical user interface by the graphical-user-interface-based inputs of a goal-state-specifying-process given by consecutive steps,

b1) a first step of receiving a choice from a user using a 3D editor, each time in doing so respectively of one different initial goal state, wherein the one different initial goal state is one of the graphical-user-interface-based inputs,

b2) a second step of generating a list depending on a chosen initial goal state number “mesi” with mesi being chosen from a number of different goal state abstraction levels, and

b3) a third step of receiving a user selection from the generated list of one different goal state abstraction level of the mesi different goal state abstraction levels, wherein the selected one different goal state abstraction level is one of the graphical-user-interface-based inputs,

c) defining a planning task, which is a combination of facts, for the specified workflow related goal state and according to the workflow related planning domain based on the facts and for at least one selected planning operator of the workflow related set of planning operators, wherein each planning operator of the at least one selected planning operator is a task capturing physical actions of the technical system workflow, and

d) by a planning approach according to the defined planning task and an execution engine

d1) checking executions of the selected planning operator iteratively in order to find a full plan comprising first operation sequences achieving a goal state of the goal states by a primary query loop, wherein in consecutive loop iterations until, by <i> executing in hardware the at least one selected planning operator, <ii> generating primary control commands due to the at least one executed selected planning operator through the execution engine, <iii> accordingly applying the primary control commands outputted via the communications interface to simulate the technical system workflow and <iv> accordingly checking system simulation through primary measurement results of measurements fed back to the execution engine, the full plan achieving the goal state is found, and

d2) checking executions of the full plan achieving the goal state iteratively by a secondary query loop, wherein in consecutive loop iterations until by <i> executing the full plan through the execution engine, <ii> generating secondary control commands due to the full plan through the execution engine, <iii> performing, by applying the secondary control commands outputted via the communications interface, a hardware execution of the technical system workflow and <iv> accordingly checking the hardware execution of the technical system workflow through secondary measurement results of measurements, the goal state is achieved.

8 . The device according to claim 7 , wherein the workflow related goal state specified by the graphical-user-interface-based inputs of the goal-state-specifying-process is given after the third step by further consecutive steps,

b4) a fourth step of sampling different goal states of selected different goal state abstraction levels and presenting the sampled different goal states to the user as presented goal states, and

b5) a fifth step of interrogating the user whether all the presented goal states are feasible, and if all the presented goal states are feasible then the different initial goal state and the selected different goal state abstraction levels are both the graphical-user-interface-based inputs; otherwise if all the presented goal states are not feasible either (“Option I”) the third step is carried out for another selected different goal state abstraction level or (“Option II”) the first step is carried out for another chosen different initial goal state.

9 . The device according to claim 7 , wherein one or more planning operators of the at least one selected planning operator are selectable for checking the executions of the at least one selected planning operator iteratively in order to find the full plan comprising the first operation sequences.

10 . The device according to claim 7 , wherein a computer into which a computer implemented tool is loadable.

11 . The device according to claim 7 , wherein the physical actions of the robotic system include one or more physical actions selected from the group consisting of torques, motions, and graspings.