IP Library › Granted Patent US 12,502,770
Granted Patent B2
US 12,502,770 · App. 18/193,901 · Granted Dec 23, 2025

Resilient multi-robot system with social learning for smart factories

Inventors: Zixiang Nie (Tampa, FL); Kwang-Cheng Chen (Tampa, FL)
Assignee: University of South Florida
B25J9/163B25J9/161B25J9/1653B25J9/1682G05B2219/39146G05B2219/39162
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,502,770
App. No.
18/193,901
Granted
Dec 23, 2025
Kind
B2
Abstract

A system and methods for operating a multi-robot system (MRS) are disclosed. In some aspects, each robot of the MRS can: determine a local system regret state belief based on local evidence obtained by the robot itself and social evidence provided by other robots in a social community, determine a local system drift state belief based on the local system regret state belief, determine a next action based on the local system regret state belief and the local system drift state belief, and execute the next action. Local system regret state belief is generally an estimation of a system regret state for the MRS. Local system drift state belief is generally an estimate of a system drift state for the MRS.

Claims (39)

1 . A method of operating a first robot in a multi-robot system, wherein the first robot and at least one additional robot of the multi-robot system form a social community in a network, the method comprising:

determining, by the first robot, a local system regret state belief based on local evidence obtained by the first robot and social evidence provided by the at least one additional robot in the social community, wherein the local system regret state belief is an estimation of a system regret state for the multi-robot system;

determining, by the first robot, a local system drift state belief based on the local system regret state belief, wherein the local system drift state belief is an estimate of a system drift state for the multi-robot system;

generating, by the first robot, an opinion vector representing a level of trust to the at least one additional robot in the social community, wherein the opinion vector is calculated from the local system drift state belief;

receiving, by a remote computing device, the opinion vector;

generating, by the remote computing device, a trust matrix based at least in part on the opinion vector and at least another received opinion vector;

responsive to detecting, by the remote computing device and based at least in part on the trust matrix, a point failure resulting in product defects, initiating a corrective action within the multi-robot system, wherein the corrective action comprises at least one of forcing a robot calibration or scheduling maintenance;

determining, by the first robot, a next action relating to a manufacturing or fabrication task based on the local system regret state belief and the local system drift state belief; and

executing, by the first robot, the next action.

2 . The method of claim 1 , further comprising:

transmitting, by the first robot, the opinion vector to the remote computing device.

3 . The method of claim 1 , wherein the local evidence comprises a least one of a local system regret state belief and/or a local system drift state belief determined by the first robot at a previous time step and measurements of a physical product obtained by the first robot.

4 . The method of claim 1 , wherein the social evidence received from the at least one additional robot indicating a state of a product being fabricated or manipulated by the multi-robot system as determined by the at least one additional robot.

5 . The method of claim 1 , wherein the social evidence comprises measurements of a physical product obtained by the at least one additional robot at a previous time step.

6 . The method of claim 1 , wherein the next action is determined using a reinforcement learning model.

7 . The method of claim 6 , wherein determining the next action is further based on a reward value provided by the remote computing device.

8 . The method of claim 1 , wherein the local system regret state belief is determined using a Bayesian network.

9 . The method of claim 1 , wherein the local system drift state belief is determined using a stochastic gradient descent (SGD) network with Huber loss.

10 . The method of claim 1 , wherein the social network is a partially connected network that shares hypervertex set {R m,n } with the physical network, G phy .

11 . A control system for a first robot in a multi-robot system, wherein the first robot and at least one additional robot of the multi-robot system form a social community in a network, the control system comprising:

a processor; and

memory having instructions stored thereon that, when executed by the processor, cause the control system to:

determine a local system regret state belief based on local evidence obtained by the first robot and social evidence provided by the at least one additional robot in the social community, wherein the local system regret state belief is an estimation of a system regret state for the multi-robot system;

determine a local system drift state belief based on the local system regret state belief, wherein the local system drift state belief is an estimate of a system drift state for the multi-robot system;

generate an opinion vector representing a level of trust to the at least one additional robot in the social community, wherein the opinion vector is calculated from the local system drift state belief, and wherein the opinion vector is transmitted to a remote computing device configured to: (i) generate a trust matrix based at least in part on the opinion vector and at least another received opinion vector, and (ii) responsive to detecting, based at least in part on the trust matrix, a point failure resulting in product defects, initiate a corrective action within the multi-robot system, wherein the corrective action comprises at least one of forcing a robot calibration or scheduling maintenance;

determine a next action for the first robot relating to a manufacturing or fabrication task based on the local system regret state belief and the local system drift state belief; and

control the first robot to execute the next action.

12 . The control system of claim 11 , wherein the local evidence comprises at least one of a local system regret state belief and/or a local system drift state belief determined by the first robot at a previous time step and measurements of a physical product obtained by the first robot.

13 . The control system of claim 11 , wherein the social evidence received from the at least one additional robot indicating a state of a product being fabricated or manipulated by the multi-robot system as determined by the at least one additional robot.

14 . The control system of claim 11 , wherein the social evidence comprises measurements of a physical product obtained by the at least one additional robot at a previous time step.

15 . The control system of claim 11 , wherein the next action is determined using a reinforcement learning model, wherein determining the next action is further based on a reward value provided by a remote device.

16 . The control system of claim 11 , wherein the local system regret state belief is determined using a Bayesian network.

17 . The control system of claim 11 , wherein the local system drift state belief is determined using a stochastic gradient descent (SGD) network with Huber loss.

18 . A non-transitory computer readable medium having instructions stored thereon that, when executed by a processor, cause a device to:

determine, for a first robot in a multi-robot system, a local system regret state belief, wherein the first robot and at least one additional robot of the multi-robot system form a social community in a network, wherein the local system regret state belief is determined based on local evidence obtained by the first robot and social evidence provided by the at least one additional robot in the social community, and wherein the local system regret state belief is an estimation of a system regret state for the multi-robot system;

determine, for the first robot, a local system drift state belief based on the local system regret state belief, wherein the local system drift state belief is an estimate of a system drift state for the multi-robot system;

generate, for the first robot, an opinion vector representing a level of trust to the at least one additional robot in the social community, wherein the opinion vector is calculated from the local system drift state belief, and wherein the opinion vector is transmitted to a remote computing device configured to: (i) generate a trust matrix based at least in part on the opinion vector and at least another received opinion vector, and (ii) responsive to detecting, based at least in part on the trust matrix, a point failure resulting in product defects, initiate a corrective action within the multi-robot system, wherein the corrective action comprises at least one of forcing a robot calibration or scheduling maintenance;

determine a next action for the first robot relating to a manufacturing or fabrication task based on the local system regret state belief and the local system drift state belief; and

control the first robot to execute the next action.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 7, 2023
From: NIE, ZIXIANG; CHEN, KWANG-CHENG
To: UNIVERSITY OF SOUTH FLORIDA
Reel/Frame 063882/0947 →
Continuity (2)
Provisional Application 63362235 · Mar 31, 2022
Related Publication 20230311312A1 · Oct 5, 2023
References Cited (51)
US 10906188B1 · Sun · 2021 [cited by examiner]
US 20200133224A1 · Popp · 2020 [cited by examiner]
US 20210107151A1 · Wouhaybi · 2021 [cited by examiner]
US 20210107152A1 · Honkote · 2021 [cited by examiner]
L. Li, K. Ota and M. Dong, “Deep Learning for Smart Industry: Efficient Manufacture Inspection System With Fog Computing,” in IEEE Transactions on Industrial Informatics, vol. 14, No. 10, pp. 4665-4673, Jun. 2018, doi: … [cited by examiner]
K. -C. Chen and H. -M. Hung, “Wireless Robotic Communication for Collaborative Multi-Agent Systems,” ICC 2019—2019 IEEE International Conference on Communications (ICC), Shanghai, China, Jul. 2019, pp. 1-7, doi: 10.1109… [cited by examiner]
K. Gokcesu and H. Gokcesu, “Generalized Huber Loss for Robust Learning and its Efficient Minimization for a Robust Statistics”, arXiv, Aug. 2021, https://doi.org/10.48550/arXiv.2108.12627 (Year: 2021). [cited by examiner]
K. -C. Chen, S. -C. Lin, J. -H. Hsiao, C. -H. Liu, A. F. Molisch and G. P. Fettweis, “Wireless Networked Multirobot Systems in Smart Factories,” in Proceedings of the IEEE, vol. 109, No. 4, pp. 468-494, Nov. 2020, doi: … [cited by examiner]
Y. Liu and G. -H. Yang, “Resilient Event-Triggered Distributed State Estimation for Nonlinear Systems Against DOS Attacks,” in IEEE Transactions on Cybernetics, vol. 52, No. 9, pp. 9076-9089, Feb. 2021, doi: 10.1109/TCY… [cited by examiner]
K. C. Chen, S. C. Lin, J. H. Hsiao, C. H. Liu, A. F. Molisch, and G. P. Fettweis, “Wireless networked multirobot systems in smart factories,” Proceedings of the IEEE, vol. 109, No. 4, pp. 468-494, 2021. [cited by applicant]
J. Wan, X. Li, H. N. Dai, A. Kusiak, M. Mart'inez-Garc'ia, and D. Li, “Artificial-intelligence-driven customized manufacturing factory: Key technologies, applications, and challenges,” Proceedings of the IEEE, vol. 109,… [cited by applicant]
F. Psarommatis, M. Danishvar, A. Mousavi, and D. Kiritsis, “Cost-based decision support system: A dynamic cost estimation of key performance indicators in manufacturing,” IEEE Transactions on Engineering Management, pp.… [cited by applicant]
S. D. Cataldo, S. Lee, E. Macii, and B. Vogel-Heuser, “Leading information and communication technologies for smart manufacturing: Facing the new challenges and opportunities of the 4th industrial revolution,” Proceedin… [cited by applicant]
A. Prorok, M. Malencia, L. Carlone, G. S. Sukhatme, B. M. Sadler, and V. Kumar, “Beyond robustness: A taxonomy of approaches towards resilient multi-robot systems,” arXiv preprint arXiv:2109.12343, 2021. [cited by applicant]
Z. Nie and K.-C. Chen, “Hypergraphical real-time multi-robot task allocation in a smart factory,” IEEE Transactions on Industrial Informatics, pp. 1-1, 2021. [cited by applicant]
Nie, Zixiang, and Kwang-Cheng Chen. “Distributed Coordination by Social Learning in the Multi-Robot Systems of a Smart Factory.” 2021 IEEE Global Communications Conference (GLOBECOM). IEEE, 2021. [cited by applicant]
T. Sun, C. Liu, B. Lian, p. Wang, and Y. Song, “Calibration for precision kinematic control of an articulated serial robot,” IEEE Transactions on Industrial Electronics, vol. 68, No. 7, pp. 6000-6009, 2021. [cited by applicant]
C.-Y. Hsu, W.-J. Chen, and J.-C. Chien, “Similarity matching of wafer bin maps for manufacturing intelligence to empower industry 3.5 for semiconductor manufacturing,” Computers & Industrial Engineering, vol. 142, p. 10… [cited by applicant]
A. Mustafa and H. Modares, “Attack analysis and resilient control design for discrete-time distributed multi-agent systems,” IEEE Robotics and Automation Letters, vol. 5, No. 2, pp. 369-376, 2020. [cited by applicant]
J. H. Hsiao and K. C. Chen, “Network analysis of collaborative cyberphysical multi-agent smart manufacturing systems : Invited paper,” in 2019 IEEE/CIC International Conference on Communications in China (ICCC). IEEE, 2… [cited by applicant]
V. Matta, A. Santos, and A. H. Sayed, “Graph learning under partial observability,” Proceedings of the IEEE, vol. 108, No. 11, pp. 2049-2066, 2020. [cited by applicant]
C. Landgraf, K. Ernst, G. Schleth, M. Fabritius, and M. F. Huber, “A hybrid neural network approach for increasing the absolute accuracy of industrial robots,” in 2021 IEEE 17th International Conference on Automation Sc… [cited by applicant]
Y. K. Teoh, S. S. Gill, and A. K. Parlikad, “Iot and fog computing based predictive maintenance model for effective asset management in industry 4.0 using machine learning,” IEEE Internet of Things Journal, pp. 1-1, 202… [cited by applicant]
D. Zhang, N. Zhang, N. Ye, J. Fang, and X. Han, “Hybrid learning algorithm of radial basis function networks for reliability analysis,” IEEE Transactions on Reliability, vol. 70, No. 3, pp. 887-900, 2021. [cited by applicant]
D. Zhang, G. Feng, Y. Shi, and D. Srinivasan, “Physical safety and cyber security analysis of multi-agent systems: A survey of recent advances,” IEEE/CAA Journal of Automatica Sinica, vol. 8, No. 2, pp. 319-333, 2021. [cited by applicant]
W. Li, H. Zhang, Y. Zhou, and Y.Wang, “Bipartite formation tracking for multi-agent systems using fully distributed dynamic edge-event-triggered protocol,” IEEE/CAA Journal of Automatica Sinica, pp. 1-7, 2022. [cited by applicant]
X. Wan, Z. Wang, M. Wu, and X. Liu, “H∞ State Estimation for Discrete-Time Nonlinear Singularly Perturbed Complex Networks Under the Round-Robin Protocol,” IEEE Transactions on Neural Networks and Learning Systems, vol.… [cited by applicant]
Y. Liu and G.-H. Yang, “Resilient event-triggered distributed state estimation for nonlinear systems against dos attacks,” IEEE Transactions on Cybernetics, pp. 1-14, 2021. [cited by applicant]
P. Duan, G. Lv, Z. Duan, and Y. Lv, “Resilient state estimation for complex dynamic networks with system model perturbation,” IEEE Transactions on Control of Network Systems, vol. 8, No. 1, pp. 135-146, 2021. [cited by applicant]
C. Chen, K. Xie, F. L. Lewis, S. Xie, and A. Davoudi, “Fully distributed resilience for adaptive exponential synchronization of heterogeneous multiagent systems against actuator faults,” IEEE Transactions on Automatic C… [cited by applicant]
H. Liang, Y. Zhou, H. Ma, and Q. Zhou, “Adaptive distributed observer approach for cooperative containment control of nonidentical networks,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 49, No. 2, … [cited by applicant]
A. Mitra, J. A. Richards, S. Bagchi, and S. Sundaram, “Resilient distributed state estimation with mobile agents: overcoming byzantine adversaries, communication losses, and intermittent measurements,” Autonomous Robots… [cited by applicant]
A. Mitra, F. Ghawash, S. Sundaram, and W. Abbas, “On the impacts of redundancy, diversity, and trust in resilient distributed state estimation,” IEEE Transactions on Control of Network Systems, vol. 8, No. 2, pp. 713-72… [cited by applicant]
H. Zhang, J. Hu, H. Liu, X. Yu, and F. Liu, “Recursive state estimation for time-varying complex networks subject to missing measurements and stochastic inner coupling under random access protocol,” Neurocomputing, vol.… [cited by applicant]
V. Bordignon, V. Matta, and A. H. Sayed, “Adaptive social learning,” IEEE Transactions on Information Theory, vol. 67, No. 9, pp. 6053-6081, 2021. [cited by applicant]
C. Henkel, J. Abbenseth, and M. Toussaint, “An optimal algorithm to solve the combined task allocation and path finding problem,” arXiv preprint arXiv:1907.10360, 2019. [cited by applicant]
M. Otte, M. J. Kuhlman, and D. Sofge, “Auctions for multi-robot task allocation in communication limited environments,” Autonomous Robots, vol. 44, No. 3, pp. 547-584, 2020. [cited by applicant]
A. Grover and J. Leskovec, “node2vec: Scalable feature learning for networks,” in Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, 2016, pp. 855-864. [cited by applicant]
H. Touzani, H. Hadj-Abdelkader, N. S'eguy, and S. Bouchafa, “Multirobot task sequencing automatic path planning for cycle time optimization: Application for car production line,” IEEE Robotics and Automation Letters, vo… [cited by applicant]
J. A. Marvel, R. Bostelman, and J. Falco, “Multi-robot assembly strategies and metrics,” ACM Comput. Surv., vol. 51, No. 1, Jan. 2018. [Online]. Available: https://doi.org/10.1145/3150225. [cited by applicant]
L. Li, K. Ota, and M. Dong, “Deep learning for smart industry: Efficient manufacture inspection system with fog computing,” IEEE Transactions on Industrial Informatics, vol. 14, No. 10, pp. 4665-4673, 2018. [cited by applicant]
A. H. Sayed, “Adaptation, learning, and optimization over networks,” Foundations and Trends in Machine Learning, vol. 7, No. Article, pp. 311-801, 2014. [cited by applicant]
K. Gokcesu and H. Gokcesu, “Generalized huber loss for robust learning and its efficient minimization for a robust statistics,” arXiv preprint arXiv:2108.12627, 2021. [cited by applicant]
S. Liu, N. Gupta, and N. H. Vaidya, “Approximate byzantine faulttolerance in distributed optimization,” in Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing, ser. PODC'21. New York, NY, USA: A… [cited by applicant]
L. Alfantoukh, Y. Ruan, and A. Durresi, “Multi-stakeholder consensus decision-making framework based on trust: A generic framework,” in 2018 IEEE 4th International Conference on Collaboration and Internet Computing (CIC… [cited by applicant]
D. Cavaliere, J. A. Morente-Molinera, V. Loia, S. Senatore, and E. Herrera-Viedma, “Collective scenario understanding in a multivehicle system by consensus decision making,” IEEE Transactions on Fuzzy Systems, vol. 28, … [cited by applicant]
Malus, Andreja, and Dominik Kozjek. “Real-time order dispatching for a fleet of autonomous mobile robots using multi-agent reinforcement learning.” CIRP annals 69.1 (2020): 397-400. [cited by applicant]
Agrawal, Akash, et al. “A multi-agent reinforcement learning framework for intelligent manufacturing with autonomous mobile robots.” Proceedings of the Design Society 1 (2021): 161-170. [cited by applicant]
Hu, Hongtao, et al. “Anti-conflict AGV path planning in automated container terminals based on multi-agent reinforcement learning.” International Journal of Production Research 61.1 (2023): 65-80. [cited by applicant]
Popper, Jens, Vassilios Yfantis, and Martin Ruskowski. “Simultaneous production and agv scheduling using multi-agent deep reinforcement learning.” Procedia CIRP 104 (2021): 1523-1528. [cited by applicant]
Yun, Won Joon, et al. “Cooperative multiagent deep reinforcement learning for reliable surveillance via autonomous multi-UAV control.” IEEE Transactions on Industrial Informatics 18.10 (2022): 7086-7096. [cited by applicant]