IP Library › Granted Patent US 12,304,477
Granted Patent B2
US 12,304,477 · App. 17/972,873 · Granted May 20, 2025

Control device, control method, and recording medium

Inventors: Masatoshi Ogawa (Yokohama, JP); Takuma Degawa (Yokohama, JP); Masakazu Mukai (Shinjuku, JP); Shigeaki Kurita (Yokohama, JP)
Assignees: TRANSTRON INC.; KOGAKUIN UNIVERSITY
B60W30/0956B60W50/0097B60W2050/0022B60W2420/403B60W2420/408B60W2420/54B60W2510/18B60W2510/20B60W2520/10B60W2520/105B60W2554/4042
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,304,477
App. No.
17/972,873
Granted
May 20, 2025
Kind
B2
Abstract

A control device calculates a first estimate value of a first monitored amount of a mobile object, from sensor information of a controlled object, using a transformed first model obtained by transformation of a first model having a multi-layered neural structure and activation functions of a ReLU structure, and predicting dynamic behavior of the mobile object. The transformed first model is a model for which weight coefficients, biases, and upper and lower limit values of input/output variables of neurons are set based on the first model and in which activation functions of the neurons are transformed using a linear inequality function having a binary variable. The control device calculates a second estimate value of a second monitored amount of the controlled object from a candidate value of an operation amount for controlling the second monitored amount. The control device determines a value of the operation amount based on a target value of the second monitored amount, the second estimate value, and the first estimate value.

Claims (54)

1. A control device, comprising:

a transformed first model for which a weight coefficient, a bias, and upper and lower limit values of an input/output variable of a neuron are set based on a first model and in which an activation function of the neuron is transformed using a linear inequality function that includes a binary variable, the first model having a multi-layered neural structure and the activation function of a rectified linear unit (ReLU) structure, the first model receiving, as input, information of a sensor equipped on a controlled object, the first model predicting dynamic behavior of a mobile object in a surroundings of the controlled object;

a memory; and

a processor coupled to the memory, the processor configured to:

obtain the information of the sensor and a target value of a monitored amount of the controlled object;

use the transformed first model to calculate an estimate value, as a future value, of a monitored amount of the mobile object, from the information of the sensor;

calculate an estimate value, as a future value, of the monitored amount of the controlled object, from a candidate value of an operation amount for controlling the monitored amount of the controlled object; and

determine a value of the operation amount for controlling the monitored amount of the controlled object, based on the target value of the monitored amount of the controlled object, the estimate value of the monitored amount of the controlled object, and the estimate value of the monitored amount of the mobile object.

2. The control device according to claim 1 , wherein

the estimate value of the monitored amount of the controlled object is calculated from the candidate value of the operation amount, using a second model that receives the operation amount of the controlled object as input and predicts dynamic behavior of the controlled object.

3. The control device according to claim 1 , wherein

the candidate value of the operation amount includes a plurality of candidate values, and

the processor:

for each of the plurality of candidate values, calculates a respective control evaluation value representing a cost for controlling the monitored amount of the controlled object, the respective control evaluation value being calculated based on the calculated estimate value of the monitored amount of the controlled object, the target value of the monitored amount of the controlled object, and the estimate value of the monitored amount of the mobile object; and

based on the calculated respective control evaluation value, determines the value of the operation amount for controlling the monitored amount of the controlled object, from among the plurality of candidate values.

4. The control device according to claim 3 , wherein

the monitored amount of the controlled object includes a path of travel and a speed of the controlled object,

the monitored amount of the mobile object includes a path of travel and a speed of the mobile object, and

the processor:

calculates the respective control evaluation value, based on an error of the estimate value of the monitored amount of the controlled object with respect to the target value of the monitored amount of the controlled object, and a positional relationship between the controlled object and the mobile object, the positional relationship being identified from the estimate value of the monitored amount of the controlled object and the estimate value of the monitored amount of the mobile object.

5. The control device according to claim 4 , wherein

the positional relationship is expressed by a presence or absence of an intersection of the estimate value of the path of travel of the controlled object and the estimate value of the path of travel of the mobile object.

6. The control device according to claim 4 , wherein

the positional relationship is expressed by an error between a position of the controlled object based on an estimate value of the speed and the path of travel of the controlled object, and a position of the mobile object based on an estimate value of the speed and the path of travel of the mobile object.

7. The control device according to claim 4 , wherein

the monitored amount of the controlled object includes fuel consumption of the controlled object,

the monitored amount of the mobile object includes fuel consumption of the mobile object, and

the processor calculates the respective control evaluation value, based on an estimate value of the fuel consumption of the controlled object and an estimate value of the fuel consumption of the mobile object.

8. The control device according to claim 4 , wherein

the processor calculates the respective control evaluation value, based on a difference of a current value of the operation amount and the candidate value of the operation amount.

9. The control device according to claim 1 , wherein

the operation amount is a steering amount, a braking amount, and an acceleration amount.

10. The control device according to claim 1 , wherein

the processor outputs the determined value of the operation amount.

11. The control device according to claim 1 , wherein

the information of the sensor includes image information of a camera equipped on the controlled object.

12. The control device according to claim 1 , wherein

the information of the sensor includes information of at least any one of light detection and ranging (LiDAR), millimeter wave radar, and an ultrasonic sensor.

13. The control device according to claim 1 , wherein

the upper and lower limit values of the input/output variable are set based on a maximum value and a minimum value of the input/output variable, obtained by providing a chirp signal or an amplitude pseudo random binary signal (APRBS) signal as the input to the first model.

14. The control device according to claim 1 , wherein

the activation function is a Leaky ReLU structure.

15. The control device according to claim 1 , wherein

the target value of the monitored amount of the controlled object is obtained from a navigation system equipped on the controlled object.

16. A control method executed by a processor, the control method comprising:

obtaining information of a sensor equipped on a controlled object and a target value of a monitored amount of the controlled object;

calculating an estimate value, as a future value, of a monitored amount of a mobile object, from the obtained information of the sensor, using a transformed first model for which a weight coefficient, a bias, and upper and lower limit values of an input/output variable of a neuron are set based on a first model and in which an activation function of the neuron is transformed using a linear inequality function that includes a binary variable, the first model having a multi-layered neural structure and the activation function of a rectified linear unit (ReLU) structure, the first model receiving, as input, the information of the sensor, the first model predicting dynamic behavior of the mobile object in a surroundings of the controlled object;

calculating an estimate value, as a future value, of the monitored amount of the controlled object, from a candidate value of an operation amount for controlling the monitored amount of the controlled object; and

determining a value of the operation amount for controlling the monitored amount of the controlled object, based on the obtained target value of the monitored amount of the controlled object, the calculated estimate value of the monitored amount of the controlled object, and the calculated estimate value of the monitored amount of the mobile object.

17. A computer-readable recording medium storing therein a control program executable by a processor, the control program comprising:

an instruction for obtaining information of a sensor equipped on a controlled object and a target value of a monitored amount of the controlled object;

an instruction for calculating an estimate value, as a future value, of a monitored amount of a mobile object, from the obtained information of the sensor, using a transformed first model for which a weight coefficient, a bias, and upper and lower limit values of an input/output variable of a neuron are set based on a first model and in which an activation function of the neuron is transformed using a linear inequality function that includes a binary variable, the first model having a multi-layered neural structure and the activation function of a rectified linear unit (ReLU) structure, the first model receiving, as input, the information of the sensor, the first model predicting dynamic behavior of the mobile object in a surroundings of the controlled object;

an instruction for calculating an estimate value, as a future value, of the monitored amount of the controlled object, from a candidate value of an operation amount for controlling the monitored amount of the controlled object; and

an instruction for determining a value of the operation amount for controlling the monitored amount of the controlled object, based on the obtained target value of the monitored amount of the controlled object, the calculated estimate value of the monitored amount of the controlled object, and the calculated estimate value of the monitored amount of the mobile object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 25, 2022
From: OGAWA, MASATOSHI; DEGAWA, TAKUMA; MUKAI, MASAKAZU; KURITA, SHIGEAKI
To: TRANSTRON INC.; KOGAKUIN UNIVERSITY
Reel/Frame 061528/0779 →
Priority Claims (2)
JP 2021-177303 · Oct 29, 2021 · national
JP 2021-177303 · Oct 29, 2021 · national
Continuity (1)
Related Publication 20230138137A1 · May 4, 2023
References Cited (11)
US 5973732A · Guthrie · 1999 [cited by examiner]
US 11577722B1 · Packer · 2023 [cited by examiner]
US 11746718B2 · Ogawa · 2023 [cited by examiner]
US 20030123703A1 · Pavlidis · 2003 [cited by examiner]
US 20070143074A1 · Gallestey et al. · 2007 [cited by applicant]
US 20190035275A1 · Nishi · 2019 [cited by examiner]
JP 2008502034A · 2008 [cited by applicant]
Joakim Haugen et al/, “Monitoring Moving Objects Using Aerial Mobile Sensors,” in IEEE Transactions on Control Systems Technology, vol. 24, No. 2, pp. 475-486, Mar. 2016, doi: 10.1109/TCST.2015.2454432. [cited by examiner]
Balázs Németh et al.; “The Design of Performance Guaranteed Autonomous Vehicle Control for Optimal Motion in Unsignalized Intersections”; Institute for Computer Science and Control; Applied Sciences MDPI; 2021, 11, 3464… [cited by applicant]
Jiachen Li et al; “Generic Tracking and Probabilistic Prediction Framework and Its Application in Autonomous Driving”; IEEE Transactions on Intelligent Transportation Systems; ARXIV.org, Cornell University; Aug. 23, 201… [cited by applicant]
Extended European Search Report dated Mar. 14, 2023 in related European Patent Application No. 22203804.4 (9 pages). [cited by applicant]