IP Library › Granted Patent US 12688476
Granted Patent B1
US 12688476 · App. 19/058,257 · Granted Jul 21, 2026

Systems and methods for advancing the restoration process for interdependent critical infrastructures

Inventors: Namrata Saha (Miami, FL); Mohammadhadi Amini (Miami, FL); Shabnam Rezapour (Miami, FL)
Assignee: The Florida International University Board of Trustees
G06Q10/0637G06N3/092G06Q10/0635G06Q10/067
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12688476
App. No.
19/058,257
Granted
Jul 21, 2026
Kind
B1
Abstract

Systems and methods are provided for advancing the restoration process for interdependent critical infrastructures (CIs) after a disaster. A modeling and solution approach can leverage the strengths of reinforcement learning (RL) to enhance the computational capability of optimization techniques to advance the restoration process. The gap can be bridged between integrative and distinct decision-making, enabling coordinated restoration planning for CIs within a decentralized decision-making context.

Claims (113)

1 . A system for advancing the restoration process for interdependent critical infrastructures (CIs) after a disaster, the system comprising:

a processor; and

a machine-readable medium in operable communication with the processor and having instructions stored thereon that, when executed by the processor, perform the following steps:

a) receiving first data from a first CI system affected by the disaster;

b) generating a first set of decisions regarding a first restoration policy for the first CI system using a first reinforcement learning (RL) module;

c) providing the first set of decisions to a coordinator module;

d) receiving second data from a second CI system affected by the disaster, the second CI system having interdependency with the first CI system while being a completely different CI system than the first CI system;

e) providing the first set of decisions from the coordinator module to a second RL module different from the first RL module;

f) generating a second set of decisions regarding a second restoration policy for the second CI system using the second RL module, the generating of the second set of decisions taking into account the first set of decisions;

g) providing the second set of decisions to the coordinator module;

h) providing the second set of decisions from the coordinator module to the first RL module;

i) generating an updated first set of decisions regarding the first restoration policy for the first CI system using the first RL module, the generating of the updated first set of decisions taking into account the second set of decisions;

j) providing the updated first set of decisions to the coordinator module;

k) providing the updated first set of decisions from the coordinator module to the second RL module;

l) Generating an updated second set of decisions regarding the second restoration policy for the second CI system using the second RL module, the generating of the updated second set of decisions taking into account the updated first set of decisions;

m) providing the updated second set of decisions to the coordinator module;

n) providing the updated second set of decisions from the coordinator module to the first RL module;

o) repeating steps i)-n) a predetermined amount of times, with each iteration of step i) taking into account the most recent updated second set of decisions and each iteration of step l) taking into account the most recent updated first set of decisions, resulting in a final first set of decisions for the first restoration policy for the first CI system and a final second set of decisions for the second restoration policy for the second CI system;

p) providing to restoration personnel the final first set of decisions and the final second set of decisions, allowing the restoration personnel to implement the first restoration policy for the first CI system and the second restoration policy for the second CI system; and

q) physically restoring the first CI system based on the first restoration policy for the first CI system, and physically restoring the second CI system based on the second restoration policy for the second CI system,

the coordinator module being configured to facilitate information exchange between the first CI system and the second CI system, thereby enabling coordinated decision-making of the first CI system and the second CI system,

the coordinator module being further configured to minimize information sharing between the first CI system and the second CI system while still allowing for the coordinated decision-making of the first CI system and the second CI system, thereby preserving privacy of the first CI system and the second CI system,

the coordinator module comprising a feasibility module (FM) and a prediction module (PM),

the FM being configured to:

store the first set of decisions and each updated iteration thereof, and the second set of decisions and each updated iteration thereof;

share the first set of decisions and each updated iteration thereof with the second RL module to inhibit the second RL module from generating an infeasible decision as part of any iteration of the second set of decisions; and

share the second set of decisions and each updated iteration thereof with the first RL module to inhibit the first RL module from generating an infeasible decision as part of any iteration of the first set of decisions, and

the PM being configured to:

generate an estimation of a maximum reward that is achievable for each iteration of the first set of decisions;

generate an estimation of a maximum reward that is achievable for each iteration of the second set of decisions;

share with the second RL the estimation of the maximum reward that is achievable for each iteration of the first set of decisions, such that the second RL optimizes an aggregated reward at each iteration of the second set of decisions by taking into account the estimation of the maximum reward that is achievable for a current iteration of the first set of decisions; and

share with the first RL the estimation of the maximum reward that is achievable for each iteration of the second set of decisions, such that the first RL optimizes the aggregated reward at each iteration of the first set of decisions by taking into account the estimation of the maximum reward that is achievable for a current iteration of the second set of decisions.

2 . The system according to claim 1 , the first CI system being a power system.

3 . The system according to claim 2 , the second CI system being a road system.

4 . The system according to claim 1 , the second CI system being a road system.

5 . The system according to claim 1 ,

the aggregated reward being a summation of rewards achieved by the first RL and the second RL.

6 . The system according to claim 5 , the aggregated reward being determined using a modified version of Bellman's equation.

7 . The system according to claim 1 , the performing of steps b), f), i), and l) comprising using an ε-greedy approach.

8 . The system according to claim 1 , the predetermined amount of times in step o) being at least 10.

9 . A method for advancing the restoration process for interdependent critical infrastructures (CIs) after a disaster, the method comprising:

a) receiving first data from a first CI system affected by the disaster;

b) generating a first set of decisions regarding a first restoration policy for the first CI system using a first reinforcement learning (RL) module;

c) providing the first set of decisions to a coordinator module;

d) receiving second data from a second CI system affected by the disaster, the second CI system having interdependency with the first CI system while being a completely different CI system than the first CI system;

e) providing the first set of decisions from the coordinator module to a second RL module different from the first RL module;

f) generating a second set of decisions regarding a second restoration policy for the second CI system using the second RL module, the generating of the second set of decisions taking into account the first set of decisions;

g) providing the second set of decisions to the coordinator module;

h) providing the second set of decisions from the coordinator module to the first RL module;

i) generating an updated first set of decisions regarding the first restoration policy for the first CI system using the first RL module, the generating of the updated first set of decisions taking into account the second set of decisions;

j) providing the updated first set of decisions to the coordinator module;

k) providing the updated first set of decisions from the coordinator module to the second RL module;

l) Generating an updated second set of decisions regarding the second restoration policy for the second CI system using the second RL module, the generating of the updated second set of decisions taking into account the updated first set of decisions;

m) providing the updated second set of decisions to the coordinator module;

n) providing the updated second set of decisions from the coordinator module to the first RL module;

o) repeating steps i)-n) a predetermined amount of times, with each iteration of step i) taking into account the most recent updated second set of decisions and each iteration of step l) taking into account the most recent updated first set of decisions, resulting in a final first set of decisions for the first restoration policy for the first CI system and a final second set of decisions for the second restoration policy for the second CI system; and

p) providing to restoration personnel the final first set of decisions and the final second set of decisions, allowing the restoration personnel to implement the first restoration policy for the first CI system and the second restoration policy for the second CI system; and

q) physically restoring the first CI system based on the first restoration policy for the first CI system, and physically restoring the second CI system based on the second restoration policy for the second CI system,

the coordinator module being configured to facilitate information exchange between the first CI system and the second CI system, thereby enabling coordinated decision-making of the first CI system and the second CI system,

the coordinator module being further configured to minimize information sharing between the first CI system and the second CI system while still allowing for the coordinated decision-making of the first CI system and the second CI system, thereby preserving privacy of the first CI system and the second CI system,

the FM being configured to:

store the first set of decisions and each updated iteration thereof, and the second set of decisions and each updated iteration thereof;

share the first set of decisions and each updated iteration thereof with the second RL module to inhibit the second RL module from generating an infeasible decision as part of any iteration of the second set of decisions; and

share the second set of decisions and each updated iteration thereof with the first RL module to inhibit the first RL module from generating an infeasible decision as part of any iteration of the first set of decisions, and

the PM being configured to:

generate an estimation of a maximum reward that is achievable for each iteration of the first set of decisions;

generate an estimation of a maximum reward that is achievable for each iteration of the second set of decisions;

share with the second RL the estimation of the maximum reward that is achievable for each iteration of the first set of decisions, such that the second RL optimizes an aggregated reward at each iteration of the second set of decisions by taking into account the estimation of the maximum reward that is achievable for a current iteration of the first set of decisions; and

share with the first RL the estimation of the maximum reward that is achievable for each iteration of the second set of decisions, such that the first RL optimizes the aggregated reward at each iteration of the first set of decisions by taking into account the estimation of the maximum reward that is achievable for a current iteration of the second set of decisions.

10 . The method according to claim 9 , the first CI system being a power system.

11 . The method according to claim 10 , the second CI system being a road system.

12 . The method according to claim 9 , the second CI system being a road system.

13 . The method according to claim 9 ,

the aggregated reward being a summation of rewards achieved by the first RL and the second RL.

14 . The method according to claim 13 , the aggregated reward being determined using a modified version of Bellman's equation.

15 . The method according to claim 9 , the performing of steps b), f), i), and l) comprising using an ε-greedy approach.

16 . The method according to claim 9 , the predetermined amount of times in step o) being at least 10.

17 . A system for advancing the restoration process for interdependent critical infrastructures (CIs) after a disaster, the system comprising:

a processor; and

a machine-readable medium in operable communication with the processor and having instructions stored thereon that, when executed by the processor, perform the following steps:

a) receiving first data from a first CI system affected by the disaster;

b) generating a first set of decisions regarding a first restoration policy for the first CI system using a first reinforcement learning (RL) module;

c) providing the first set of decisions to a coordinator module;

d) receiving second data from a second CI system affected by the disaster, the second CI system having interdependency with the first CI system while being a completely different CI system than the first CI system;

e) providing the first set of decisions from the coordinator module to a second RL module different from the first RL module;

f) generating a second set of decisions regarding a second restoration policy for the second CI system using the second RL module, the generating of the second set of decisions taking into account the first set of decisions;

g) providing the second set of decisions to the coordinator module;

h) providing the second set of decisions from the coordinator module to the first RL module;

i) generating an updated first set of decisions regarding the first restoration policy for the first CI system using the first RL module, the generating of the updated first set of decisions taking into account the second set of decisions;

j) providing the updated first set of decisions to the coordinator module;

k) providing the updated first set of decisions from the coordinator module to the second RL module;

l) Generating an updated second set of decisions regarding the second restoration policy for the second CI system using the second RL module, the generating of the updated second set of decisions taking into account the updated first set of decisions;

m) providing the updated second set of decisions to the coordinator module;

n) providing the updated second set of decisions from the coordinator module to the first RL module;

o) repeating steps i)-n) a predetermined amount of times, with each iteration of step i) taking into account the most recent updated second set of decisions and each iteration of step l) taking into account the most recent updated first set of decisions, resulting in a final first set of decisions for the first restoration policy for the first CI system and a final second set of decisions for the second restoration policy for the second CI system;

p) providing to restoration personnel the final first set of decisions and the final second set of decisions, allowing the restoration personnel to implement the first restoration policy for the first CI system and the second restoration policy for the second CI system; and

q) physically restoring the first CI system based on the first restoration policy for the first CI system, and physically restoring the second CI system based on the second restoration policy for the second CI system,

the coordinator module being configured to facilitate information exchange between the first CI system and the second CI system, thereby enabling coordinated decision-making of the first CI system and the second CI system,

the coordinator module being further configured to minimize information sharing between the first CI system and the second CI system while still allowing for the coordinated decision-making of the first CI system and the second CI system, thereby preserving privacy of the first CI system and the second CI system,

the first CI system being a power system,

the second CI system being a road system,

the coordinator module comprising a feasibility module (FM) and a prediction module (PM),

the FM being configured to:

store the first set of decisions and each updated iteration thereof, and the second set of decisions and each updated iteration thereof;

share the first set of decisions and each updated iteration thereof with the second RL module to inhibit the second RL module from generating an infeasible decision as part of any iteration of the second set of decisions; and

share the second set of decisions and each updated iteration thereof with the first RL module to inhibit the first RL module from generating an infeasible decision as part of any iteration of the first set of decisions,

the PM being configured to:

generate an estimation of a maximum reward that is achievable for each iteration of the first set of decisions;

generate an estimation of a maximum reward that is achievable for each iteration of the second set of decisions;

share with the second RL the estimation of the maximum reward that is achievable for each iteration of the first set of decisions, such that the second RL optimizes an aggregated reward at each iteration of the second set of decisions by taking into account the estimation of the maximum reward that is achievable for a current iteration of the first set of decisions; and

share with the first RL the estimation of the maximum reward that is achievable for each iteration of the second set of decisions, such that the first RL optimizes the aggregated reward at each iteration of the first set of decisions by taking into account the estimation of the maximum reward that is achievable for a current iteration of the second set of decisions,

the aggregated reward being a summation of rewards achieved by the first RL and the second RL,

the aggregated reward being determined using a modified version of Bellman's equation, and the performing of steps b), f), i), and l) comprising using an ε-greedy approach.