Granted Patent B2

US 12,544,923 · App. 17/799,984 · Granted Feb 10, 2026

Inventors: Hisaya Wakayama (Tokyo, JP); Masatsugu Ogawa (Tokyo, JP); Takehiro Itou (Tokyo, JP); Mineto Satoh (Tokyo, JP); Hiroyuki Oyama (Tokyo, JP); Nobuharu Kami (Tokyo, JP)

Assignee: NEC CORPORATION

B25J9/1661G05B19/4182G05B19/41895G05B2219/39102

Abstract

Claims (70)

1 . A control device comprising

at least one memory storing instructions; and

at least one processor configured to execute the instructions to:

generate time step-based operation sequences indicating operations to be executed by a robot executing a task and peripheral equipment that delivers or receives an object relating to the task to or from the robot, based on:

robot operation information indicating operation characteristics of the robot executing the task; and

peripheral equipment information indicating operation characteristics of the peripheral equipment that delivers or receives the object relating to the task to or from the robot;

generate control signals to control the robot and the peripheral equipment based on:

the operation sequences; and

states of the robot and the peripheral equipment measured during execution of the operation sequences;

provide the control signals to the robot and the peripheral equipment;

predict states of the robot and the peripheral equipment according to the operation sequences;

recognize an actual progress degree of the operation sequences based on a number of completed subtasks;

determine, based on the actual progress degree and an expected progress degree of the operation sequences at a current time step, whether or not the measured states match the predicted states;

regenerate, upon determining that the measured states do not match predicted states, a control signal to control the robot and/or the peripheral equipment to adjust an operation timing of at least one of the robot or the peripheral equipment, and provide the regenerated control signal to the robot and/or the peripheral equipment;

convert the task into a logical formula based on a temporal logic, by:

setting a number of time steps for completing the task;

determining, from a target logical formula, a plurality of candidate combinations of propositions, the propositions, in each candidate combination, each representing a state at each time step, such that each candidate combination satisfies the target logical formula with the set number of times, each candidate combination being a candidate of a logical formula representing a sequence of operations to be instructed to the robot;

generating a time step logical formula in which the candidate combinations are combined; and

generating, based on the time step logical formula, sequences of subtasks to be executed by the robot and the peripheral equipment as the operation sequences, respectively.

2 . The control device according to claim 1 , wherein the at least one processor is further configured to execute the instructions to:

predict a state of the robot after elapse of a predetermined time in a workspace; and

generate the operation sequences based on the predicted state, the robot operation information, and the peripheral equipment operation information.

3 . The control device according to claim 2 , wherein the predetermined time is set to a predicted calculation time required for calculation of the operation sequences.

4 . The control device according to claim 1 , wherein the robot operation information includes at least one of:

operation limit information relating to an operation limit of the robot;

subtask information indicating subtasks executable by the robot; and

abstract model information relating to a model in which dynamics relating to the robot are abstracted,

wherein the peripheral equipment operation information includes at least one of:

operation limit information relating to an operation limit of the peripheral equipment;

subtask information indicating subtasks executable by the peripheral equipment; and

abstract model information relating to a model in which dynamics relating to the peripheral equipment are abstracted.

5 . The control device according to claim 4 , wherein the abstract model information included in the robot operation information includes information indicating a condition of switching the dynamics relating to the robot, and

wherein the abstract model information included in the peripheral equipment operation information includes information indicating a condition of switching the dynamics relating to the peripheral equipment.

6 . The control device according to claim 1 , wherein the peripheral equipment is a conveyor configured to convey the object, and

wherein the robot is a robot at least configured to pick up the object.

7 . The control device according to claim 1 , wherein the at least one processor is configured to execute the instructions to:

display an image obtained by capturing an image of a workspace of the robot on a display device and accept an input specifying the object or a type of the object; and

generate the operation sequences based on the input.

8 . A control method performed by a computer and comprising:

generating time step-based operation sequences indicating operations to be executed by a robot executing a task and peripheral equipment that delivers or receives an object relating to the task to or from the robot, based on:

robot operation information indicating operation characteristics of the robot executing the task; and

peripheral equipment information indicating operation characteristics of the peripheral equipment that delivers or receives the object relating to the task to or from the robot;

generating control signals to control the robot and the peripheral equipment based on:

the operation sequences; and

states of the robot and the peripheral equipment measured during execution of the operation sequences;

providing the control signals to the robot and the peripheral equipment;

predicting states of the robot and the peripheral equipment according to the operation sequences;

recognizing an actual progress degree of the operation sequences based on a number of completed subtasks;

determining, based on the actual progress degree and an expected progress degree of the operation sequences at a current time step, whether or not the measured states match the predicted states;

regenerating, upon determining that the measured states do not match predicted states, a control signal to control the robot and/or the peripheral equipment to adjust an operation timing of at least one of the robot or the peripheral equipment, and provide the regenerated control signal to the robot and/or the peripheral equipment;

converting the task into a logical formula based on a temporal logic, by:

setting a number of time steps for completing the task;

determining, from a target logical formula, a plurality of candidate combinations of propositions, the propositions, in each candidate combination, each representing a state at each time step, such that each candidate combination satisfies the target logical formula with the set number of times, each candidate combination being a candidate of a logical formula representing a sequence of operations to be instructed to the robot;

generating a time step logical formula in which the candidate combinations are combined; and

generating, based on the time step logical formula, sequences of subtasks to be executed by the robot and the peripheral equipment as the operation sequences, respectively.

9 . A non-transitory computer readable storage medium storing a program executable by a computer to:

generate time step-based operation sequences indicating operations to be executed by a robot executing a task and peripheral equipment that delivers or receives an object relating to the task to or from the robot, based on:

robot operation information indicating operation characteristics of the robot executing the task; and

peripheral equipment information indicating operation characteristics of the peripheral equipment that delivers or receives the object relating to the task to or from the robot;

generate control signals to control the robot and the peripheral equipment based on:

the operation sequences; and

states of the robot and the peripheral equipment measured during execution of the operation sequences; provide the control signals to the robot and the peripheral equipment; predict states of the robot and the peripheral equipment according to the operation sequences;

recognize an actual progress degree of the operation sequences based on a number of completed subtasks;

determine, based on the actual progress degree and an expected progress degree of the operation sequences at a current time step, whether or not the measured states match the predicted states;

regenerate, upon determining that the measured states do not match predicted states, a control signal to control the robot and/or the peripheral equipment to adjust an operation timing of at least one of the robot or the peripheral equipment, and provide the regenerated control signal to the robot and/or the peripheral equipment;

convert the task into a logical formula based on a temporal logic, by:

setting a number of time steps for completing the task;

determining, from a target logical formula, a plurality of candidate combinations of propositions, the propositions, in each candidate combination, each representing a state at each time step, such that each candidate combination satisfies the target logical formula with the set number of times, each candidate combination being a candidate of a logical formula representing a sequence of operations to be instructed to the robot;

generating a time step logical formula in which the candidate combinations are combined; and

generating, based on the time step logical formula, sequences of subtasks to be executed by the robot and the peripheral equipment as the operation sequences, respectively.

Assignments (1)

Continuity (1)

References Cited (26)