IP Library › Granted Patent US 12,248,286
Granted Patent B2
US 12,248,286 · App. 17/345,913 · Granted Mar 11, 2025

Inferring device, training device, inferring method, and training method

Inventors: Kento Kawaharazuka (Tokyo-to, JP); Toru Ogawa (Tokyo-to, JP)
Assignee: Preferred Networks, Inc.
G05B13/048B25J9/161G05B13/047G06N3/084G06N5/04G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,248,286
App. No.
17/345,913
Granted
Mar 11, 2025
Kind
B2
Abstract

To infer dynamic control information on a controlled object. An inferring device includes one or more memories and one or more processors. The one or more processors are configured to: input at least data about a state of a controlled object and time-series control information for controlling the controlled object, into a network trained by machine learning; acquire predicted data about a future state of the controlled object controlled based on the time-series control information via the network into which the data about the state of the controlled object and the time-series control information have been input; and output new time-series control information for controlling the controlled object to bring the future state of the controlled object into a target state based on the predicted data acquired via the network.

Claims (89)

1. An inferring device comprising:

one or more memories; and

one or more processors configured to:

input at least data about a state of a controlled object and time-series control information for controlling the controlled object, into a trained network that has been trained by machine learning;

acquire predicted data about a future state of the controlled object controlled based on the time-series control information, via the trained network into which the data about the state of the controlled object and the time-series control information have been input; and

output, through a backpropagation on at least a part of the trained network based on at least the predicted data about the future state of the controlled object acquired from the trained network, new time-series control information for controlling the controlled object to bring the future state of the controlled object into a target state, wherein:

the trained network comprises

a second trained network configured to output data based on at least the data about the state of the controlled object and the time-series control information, and

a third trained network configured to output the predicted data based on the data output from the second trained network; and

the one or more processors are configured to

output, via the second trained network, the data based on at least the data about the state of the controlled object and the time-series control information, and

output, via the third trained network, the predicted data based on the data output via the second trained network.

2. The inferring device according to claim 1 , wherein

the time-series control information to be input into the trained network and the new time-series control information to be output are the same kind of information.

3. The inferring device according to claim 1 , wherein

the one or more processors are configured to calculate the new time-series control information based on the predicted data and the time-series control information input into the trained network.

4. The inferring device according to claim 1 , wherein

the one or more processors are configured to

calculate first data based on the predicted data and on the target state of the controlled object,

calculate second data to be used for updating at least part of the time-series control information input into the trained network, from the calculated first data via the at least part of the trained network, and

update the at least part of the time-series control information input into the trained network based on the calculated second data, and output the new time-series control information including the updated part.

5. The inferring device according to claim 1 , wherein

the time-series control information input into the trained network and the new time-series control information output based on the predicted data acquired via the trained network are time-series control information for predetermined frames for controlling the controlled object.

6. The inferring device according to claim 1 , wherein

the control information for at least part of the frames of the new time-series control information is control information obtained by updating control information for part of the frames of the time-series control information input into the trained network based on the predicted data.

7. The inferring device according to claim 1 , wherein

the one or more processors are configured to output the new time-series control information to a control device for controlling the controlled object.

8. The inferring device according to claim 1 , further comprising

the trained network trained by machine learning.

9. The inferring device according to claim 1 , wherein

the time-series control information is generated based on the new time-series control information output a unit time period ago.

10. The inferring device according to claim 1 , wherein

the one or more processors are configured to execute backpropagation based on the predicted data for the third trained network, and output the time-series control information to bring the state of the controlled object into the target state.

11. The inferring device according to claim 1 , wherein

the one or more processors are configured to

calculate a loss between the output data from the third trained network and the target state, and

output the new time-series control information via at least the third trained network based on a gradient of the loss.

12. The inferring device according to claim 1 , wherein:

the trained network comprises

a first trained network configured to extract a feature amount of the controlled object from the state of the controlled object,

the second trained network being a trained network configured to output data based on the feature amount of the controlled object and on the time-series control information; and

the one or more processors are configured to

extract the feature amount of the controlled object from the state of the controlled object via the first trained network, and

output, as the data based on at least the data about the state of the controlled object and the time-series control information, data based on the feature amount of the controlled object and the time-series control information via the second trained network.

13. The inferring device according to claim 1 , wherein:

the second trained network is a trained network configured to output data obtained by mixing the data about the information on the controlled object, and the time-series control information given for controlling the state of the controlled object within a predetermined time period from the state of the controlled object; and

the third trained network is a trained network configured to output the data about the state of the controlled object when controlled by the time-series control information within the predetermined time period.

14. The inferring device according to claim 1 , wherein:

the second trained network is a trained network configured to output data obtained by mixing the data about the information on the controlled object, and the time-series control information given for controlling the state of the controlled object within a predetermined time period from the state of the controlled object and generated based on the new time-series control information output a unit time period ago; and

the third trained network is a trained network configured to output the data about the state of the controlled object when controlled by the time-series control information within the predetermined time period.

15. The inferring device according to claim 1 , wherein the one or more processors are configured to

perform, for each of a plurality of pieces of the time-series control information, the inputting and acquiring,

select a piece of the time-series control information from the plurality of pieces of the time-series control information, based on each predicted data about the future state of the controlled object acquired by the performing,

execute backpropagation for the selected time-series control information, and

update the selected time-series control information based on data acquired via the backpropagation.

16. The inferring device according to claim 1 , wherein

the data showing the state of the controlled object is image information on the controlled object and speed information on the controlled object.

17. The inferring device according to claim 1 , wherein

the second trained network outputs the time-series control information mixed with a feature amount of the controlled object.

18. The inferring device according to claim 1 , wherein

the one or more processors are configured to output the new time-series control information for controlling the state of the controlled object in real time.

19. The inferring device according to claim 1 , wherein the outputting of the new time-series control information includes:

acquiring, through the backpropagation on at least the part of the trained network based on at least (i) the predicted data of the controlled object acquired from the trained network and (ii) a target data of the predicted data, one or more gradients for updating at least a part of the time-series control information having been input into the trained network; and

updating at least the part of the time-series control information based on at least the one or more gradients acquired through the backpropagation, to obtain the new time-series control information.

20. The inferring device according to claim 1 , wherein the backpropagation of the outputting of the new time-series control information does not update parameters of the trained network.

21. The inferring device according to claim 1 , wherein the trained network is fully trained.

22. The inferring device according to claim 1 , wherein the outputting the new time-series control information is performed without updating parameters of the trained network.

23. The inferring device according to claim 1 , wherein the second trained network corresponds an intermediate layer.

24. An inferring method comprising:

inputting at least data about a state of a controlled object and time-series control information for controlling the controlled object, into a trained network that has been trained by machine learning;

acquiring predicted data about a future state of the controlled object controlled based on the time-series control information, via the trained network; and

outputting, through a backpropagation on at least a part of the trained network based on at least the predicted data about the future state of the controlled object acquired from the trained network, new time-series control information for controlling the controlled object to bring the future state of the controlled object into a target state,

wherein the trained network comprises

a second trained network configured to output data based on at least the data about the future state of the controlled object and the time-series control information, and

a third trained network configured to output the predicted data based on the data output from the second trained network; and

the one or more processors are configured to

output, via the second trained network, the data based on at least the data about the future state of the controlled object and the time-series control information, and

output, via the third trained network, the predicted data from the data based on the data output via the second trained network.

25. The inferring method according to claim 24 , wherein the second trained network corresponds to an intermediate layer.

26. An inferring system comprising:

one or more memories storing a trained network for outputting, based on inputting at least a current state of a controlled object and control information for controlling the controlled object into the trained network, data indicating a predicted future state of the controlled object; and

one or more processors configured to perform a backpropagation of at least a part of the trained network based on the output data output from the trained network, and output a new control information for the controlled object based on the control information inputted into the trained network and data acquired by the backpropagation,

wherein the trained network comprises

a second trained network configured to output data based on at least the data about the state of the controlled object and the control information for controlling the controlled object, and

a third trained network configured to output the data indicating the predicted future state of the controlled object based on the output from the second trained network; and

the one or more processors are configured to

output, via the second trained network, the data, and

output, via the third trained network, the data indicating the predicted future state of the controlled object.

27. The inferring system according to claim 26 , wherein the second trained network corresponds to an intermediate layer.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2021
From: KAWAHARAZUKA, KENTO; OGAWA, TORU
To: PREFERRED NETWORKS, INC.
Reel/Frame 057921/0799 →
Priority Claims (1)
JP 2018-232760 · Dec 12, 2018 · national
Continuity (2)
Continuation PCTJP2019045331 · Nov 19, 2019
Related Publication 20210302926A1 · Sep 30, 2021
References Cited (31)
US 10300600B2 · Yamaoka et al. · 2019 [cited by applicant]
US 10603797B2 · Ozaki et al. · 2020 [cited by applicant]
US 20140309836A1 · Ollis · 2014 [cited by examiner]
US 20150227676A1 · Robles · 2015 [cited by examiner]
US 20180056520A1 · Ozaki et al. · 2018 [cited by applicant]
US 20180207797A1 · Yamaoka et al. · 2018 [cited by applicant]
US 20180354126A1 · Ueda · 2018 [cited by examiner]
US 20190228587A1 · Mordvintsev · 2019 [cited by examiner]
US 20200016742A1 · Ishikawa et al. · 2020 [cited by applicant]
US 20200292102A1 · Ducceschi · 2020 [cited by examiner]
US 20200373726A1 · Wetzel · 2020 [cited by examiner]
US 20210107142A1 · Solowjow · 2021 [cited by examiner]
US 20210247744A1 · Fahrenkopf · 2021 [cited by examiner]
JP 2018030185A · 2018 [cited by applicant]
JP 2018118353A · 2018 [cited by applicant]
WO WO2018096902A1 · 2018 [cited by applicant]
Hecht-Nielsen, “Theory of the backpropagation neural network,” International 1989 Joint Conference on Neural Networks, Washington, DC, USA, 1989, pp. 593-605 vol. 1, doi: 10.1109/IJCNN.1989.118638. (Year: 1989). [cited by examiner]
Alex X. Lee et al., “Learning Force-Based Manipulation of Deformable Objects from Multiple Demonstrations”, 2015 IEEE International Conference on Robotics and Automation (ICRA), 2015 177-184. [cited by applicant]
Arunkumar Byravan et al., “SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Control”, 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018, 3339-3346. [cited by applicant]
Arunkumar Byravan et al., “SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Planning and Control”, arXiv: 1710.00489 v1 [cs.RO] Oct. 2, 2017. [cited by applicant]
Christof Elbrechter et al., “Folding Paper with Anthropomorphic Robot Hands using Real-Time Physics-Based Modeling”, 2012 12th IEEE-RAS International Conference on Humanoid Robots, 2012, 210-215. [cited by applicant]
Daisuke Tanaka et al., “EMD Net: An Encode-Manipulate-Decode Network for Cloth Manipulation”, IEEE Robotics and Automation Letters, vol. 3, No. 3, 2018, 1771-1778. [cited by applicant]
Gunilla Borgefors, “Hierarchical Chamfer Matching: A Parametric Edge Matching Algorithm”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 10, No. 6, 1988, 849-865. [cited by applicant]
Johannes Englsberger et al., “Overview of the torque-controlled humanoid robot TORO”, 2014 14th IEEE-RAS International Conference on Humanoid Robots (Humanoids), 2014, 916-923. [cited by applicant]
Mitul Saha et al., “Manipulation Planning for Deformable Linear Objects”, IEEE Transactions on Robotics, vol. 23, No. 6, 2007, 1141-1150. [cited by applicant]
P. Jimenez, “Survey on model-based manipulation planning of deformable objects”, Robotics and Computer-Integrated Manufacturing, vol. 28, No. 2, 2012, 154-163. [cited by applicant]
Pin-Chu Yang et al., “Repeatable Folding Task by Humanoid Robot Worker Using Deep Learning”, IEEE Robotics and Automation Letters, vol. 2, No. 2, 2017, 397-403. [cited by applicant]
T. M. Caldwell et al., “Optimal Parameter Identification for Discrete Mechanical Systems with Application to Flexible Object Manipulation”, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2014,… [cited by applicant]
Yuji Yamakawa et al., “Motion Planning for Dynamic Folding of a Cloth with Two High-speed Robot Hands and Two High-speed Sliders”, 2011 IEEE International Conference on Robotics and Automation, 2011, 5486-5491. [cited by applicant]
Yuji Yamakawa et al., “Motion Planning for Dynamic Knotting of a Flexible Rope with a High-speed Robot Arm”, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010, 49-54. [cited by applicant]
Zhe Hu et al., “Three-Dimensional Deformable Object Manipulation Using Fast Online Gaussian Process Regression”, IEEE Robotics and Automation Letters, vol. 3, No. 2, 2018, 979-986. [cited by applicant]