IP Library › Granted Patent US 12,370,684
Granted Patent B2
US 12,370,684 · App. 17/799,721 · Granted Jul 29, 2025

Control device, control method and storage medium

Inventors: Mineto Satoh (Tokyo, JP); Nobuharu Kami (Tokyo, JP); Masatsugu Ogawa (Tokyo, JP); Hisaya Wakayama (Tokyo, JP); Takehiro Itou (Tokyo, JP); Hiroyuki Oyama (Tokyo, JP)
Assignee: NEC CORPORATION
B25J9/1656G05B2219/50391
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,370,684
App. No.
17/799,721
Granted
Jul 29, 2025
Kind
B2
Abstract

The control device 1A mainly includes a determination means 15 A, an abstract state setting means 16 A, and a sequence generation means 17 A. The determination means 15 A determines, based on at least one of environment information relating to environment observed in a workspace of a controlled device to be controlled, state information relating to a state of the controlled device, and stored information that is stored information relating to the objective task to be executed by the controlled device, whether or not the objective task can be completed. The abstract state setting means 16 A sets, when determined that the objective task cannot be completed, an abstract state in the workspace based on at least one of the environment information or the stored information. The sequence generation means 17 A generates, based on the abstract state and the objective task, a sequence of subtasks to be executed by the controlled device.

Claims (66)

1. A control device comprising:

at least one memory storing instructions; and

at least one processor configured to execute the instructions to:

upon receiving a designation of an objective task to be executed by a controlled device, and in advance of setting a sequence of subtasks into which an objective task is decomposed, determine,

based on at least one of environment information relating to an environment observed in a workspace of the controlled device, state information relating to a state of the controlled device, and stored information that relating to the objective task to be executed by the controlled device,

whether or not the objective task can be completed;

set, if it is determined that the objective task cannot be completed, an abstract state in the workspace as propositions to express the objective task as a logical formula based on at least one of the environment information or the stored information;

generate, based on the abstract state and the objective task, the sequence of subtasks to be executed by the controlled device; and

control the controlled device by transmitting a control signal including the sequence of subtasks to the controlled device.

2. The control device according to claim 1 ,

wherein, at a time when the objective task is designated, the at least one processor is configured to execute the instructions to set, based on at least one of the environment information or the stored information, a first abstract state that is an abstract state in the workspace,

wherein, at the time when the objective task is designated, the at least one processor is configured to execute the instructions to generate, based on the first abstract state and the objective task, a first sequence of subtasks to be executed by the controlled device, and

wherein, during execution of the first sequence by the controlled device, the at least one processor is configured to execute the instructions to determine that the objective task cannot be completed if it is determined,

based on at least one of

the environment information,

the state information, and

the stored information,

that at least one of subtasks included in the first sequence cannot be executed.

3. The control device according to claim 2 ,

wherein the stored information includes subtask execution condition information indicating an execution condition of each of the subtasks included in the first sequence, and

wherein the at least one processor is configured to execute the instructions to determine, based on the subtask execution condition information, a feasibility of each of the subtasks included in the first sequence.

4. The control device according to claim 2 ,

wherein, if it is determined that the at least one of the subtasks cannot be executed, the at least one processor is configured to execute the instructions to set,

based on the environment information at a time of determination that the at least one of the subtasks cannot be executed,

a second abstract state that is an abstract state in the workspace, and

wherein the at least one processor is configured to execute the instructions to generate, based on the second abstract state and the objective task, a second sequence of subtasks to be executed by the controlled device.

5. The control device according to claim 1 ,

wherein the at least one processor is configured to execute the instructions to determine that the objective task cannot be completed if it is determined that an accuracy or an information amount of the environment information observed at a time when the objective task is designated does not reach an accuracy or an amount of information necessary for executing the objective task.

6. The control device according to claim 5 ,

wherein the at least one processor is configured to execute the instructions to set,

based on at least one of the environment information or the stored information which are acquired at the time when the objective task is designated,

a temporary abstract state that is the abstract state to be temporarily used, and

wherein the at least one processor is configured to execute the instructions to generate,

based on the temporary abstract state and the objective task,

a temporary sequence that is a temporary sequence of subtasks to be executed by the controlled device.

7. The control device according to claim 6 ,

wherein the at least one processor is configured to execute the instructions to set, based on the environment information at the time when the objective task is designated, the temporary abstract state within a range in which the accuracy or the information amount of the environment information reaches the accuracy or the amount of information necessary for executing the objective task.

8. The control device according to claim 6 ,

wherein the at least one processor is configured to execute the instructions to set, based on the stored information, the temporary abstract state within a range in which the accuracy or the information amount of the environment information does not reach the accuracy or the amount of information necessary for executing the objective task.

9. The control device according to claim 8 ,

wherein the stored information is task execution history information indicative of a past execution history of the objective task or map information indicative of a map in the workspace, and

wherein the at least one processor is configured to execute the instructions to set the temporal abstract state based on at least one of the task execution history information or the map information if it is determined that the accuracy or the information amount of the environment information does not reach the accuracy or the amount of information necessary for executing the objective task.

10. The control device according to claim 6 ,

wherein the at least one processor is configured to execute the instructions to set, based on the environment information, the abstract state that is an abstract state in the workspace in a case of acquiring the environment information having an accuracy and information amount necessary for the executing the objective task, and

wherein the at least one processor is configured to execute the instructions to generate, based on the abstract state and the objective task, a second sequence of subtasks to be executed by the controlled device.

11. The control device according to claim 1 ,

wherein the at least one processor is configured to execute the instructions to:

convert the objective task into a logical formula based on temporal logic;

generate a time step logical formula that is a logical formula representing a state at each time step for executing the objective task,

generate the sequence based on the time step logical formula.

12. The control device according to claim 1 ,

wherein the at least one processor is configured to execute the instructions to determine that the objective task cannot be completed, upon determining that there is a blind spot from an observation device which supplies the environment information.

13. A control method performed by a computer and comprising:

upon receiving a designation of an objective task to be executed by a controlled device, and in advance of setting a sequence of subtasks into which an objective task is decomposed, determining,

based on at least one of environment information relating to an environment observed in a workspace of the controlled device, state information relating to a state of the controlled device, and stored information that relating to the objective task to be executed by the controlled device,

whether or not the objective task can be completed;

setting, when it is determined that the objective task cannot be completed, an abstract state in the workspace as propositions to express the objective task as a logical formula based on at least one of the environment information or the stored information;

generating, based on the abstract state and the objective task, the sequence of subtasks to be executed by the controlled device; and

controlling the controlled device by transmitting a control signal including the sequence of subtasks to the controlled device.

14. A non-transitory computer readable storage medium storing a program executable by a computer to:

upon receiving a designation of an objective task to be executed by a controlled device, and in advance of setting a sequence of subtasks into which an objective task is decomposed, determine,

based on at least one of environment information relating to an environment observed in a workspace of the controlled device, state information relating to a state of the controlled device, and stored information that relating to the objective task to be executed by the controlled device,

whether or not the objective task can be completed;

set, if it is determined that the objective task cannot be completed, an abstract state in the workspace as propositions to express the objective task as a logical formula based on at least one of the environment information or the stored information;

generate, based on the abstract state and the objective task, the sequence of subtasks to be executed by the controlled device; and

control the controlled device by transmitting a control signal including the sequence of subtasks to the controlled device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 15, 2022
From: SATOH, MINETO; KAMI, NOBUHARU; OGAWA, MASATSUGU; WAKAYAMA, HISAYA; ITOU, TAKEHIRO; OYAMA, HIROYUKI
To: NEC CORPORATION
Reel/Frame 060804/0711 →
Continuity (1)
Related Publication 20230082482A1 · Mar 16, 2023
References Cited (16)
US 9689696B1 · Russell · 2017 [cited by examiner]
US 9989371B1 · Russell · 2018 [cited by examiner]
US 11577741B1 · Reschka · 2023 [cited by examiner]
US 20020169733A1 · Peter, II · 2002 [cited by applicant]
US 20100206651A1 · Nagasaka · 2010 [cited by applicant]
US 20150331416A1 · Feniello et al. · 2015 [cited by applicant]
US 20180085922A1 · Ooba · 2018 [cited by applicant]
US 20190143524A1 · Takahashi et al. · 2019 [cited by applicant]
JP 2004283958A · 2004 [cited by applicant]
JP 2005508761A · 2005 [cited by applicant]
JP 2010142940A · 2010 [cited by applicant]
JP 2010188471A · 2010 [cited by applicant]
JP 2018051684A · 2018 [cited by applicant]
JP 2019084664A · 2019 [cited by applicant]
International Search Report for PCT Application No. PCT/JP2020/007418, mailed on Apr. 7, 2020. [cited by applicant]
Japanese Office Action for JP Application No. 2022-502355, mailed on May 9, 2023 with English Translation. [cited by applicant]