Distributed reinforcement learning and consensus control of energy systems
Disclosed herein are methods, systems, and devices for utilizing distributed reinforcement learning and consensus control to most effectively generate and utilize energy. In some embodiments, individual turbines within a wind farm may communicate to reach a consensus as to the desired yaw angle based on the wind conditions.
1. A device comprising:
at least one processor configured to:
receive a wind measurement that represents a characteristic of wind as measured at a first location;
receive one or more T2 consensus wind estimates that each represents an estimate of the value of the characteristic of wind as determined from the perspective of a respective second location;
determine, based on the wind measurement and the one or more T2 consensus wind estimates, using an augmented Lagrangian method, a T1 consensus wind estimate that represents an estimate of the value of the characteristic of wind as determined from the perspective of the first location,
receive one or more T2 comparison factors that each represents a respective amount of offset as determined from the perspective of the respective second location, based on a previous value of the one or more T2 comparison factors and a previous value of the one or more T2 consensus wind estimates;
determine, based on the one or more T2 comparison factors, a previous value of the T1 consensus wind estimate, and a previous value of one or more T1 comparison factors that each represents an amount of offset as determined from the perspective of the first location with respect to the respective second location;
determine the T1 consensus wind estimate based further on:
the wind measurement,
the previous value of the T1 consensus wind estimate;
the one or more T1 comparison factors;
a Lagrangian penalty factor that represents a weighing of the second location relative to the first location, and
a total number of grouped wind turbines; and
cause a wind turbine to adjust an operating parameter based on the T1 consensus wind estimate.
2. The device of claim 1 , wherein the augmented Lagrangian methods comprise an alternating direction method of multipliers (ADMM) method.
3. The device of claim 1 , wherein determining the T1 consensus wind estimate comprises determining (x i m+1 ) using
2
x
m
e
a
s
u
r
e
-
ρ
Σ
j
∈
𝒩
(
i
)
(
-
z
i
j
m
+
u
i
j
m
)
ρ
N
t
u
r
b
s
+
2
wherein:
x measure represents the wind measurement,
z ij m represents one or more estimated T2 consensus wind estimates,
u ij m represents the one or more T1 comparison factors,
ρ represents the Lagrangian penalty factor, and
N turbs represents the total number of grouped locations, wherein:
determining, for each respective second location, the one or more estimated T2 consensus wind estimates (z ij m ) comprises determining
θ
(
x
i
m
+
u
i
j
m
-
1
)
+
(
1
-
θ
)
(
x
j
m
+
u
j
i
m
-
1
)
wherein:
θ represents a scaling factor,
x i m represents the previous value of the T1 consensus wind estimate,
u ij m−1 represents a previous value of the one or more T1 comparison factors,
x j m represents the one or more T2 consensus wind estimates, and
u ji m−1 represents the one or more T2 comparison factors; and
determining, for each respective second location, the one or more T1 comparison factors (u ji m ) comprises determining u ij m−1 +(x i m −x ij m ) wherein:
u ij m−1 represents the previous value of the one or more T1 comparison factors,
x i m represents the previous value of the T1 consensus wind estimate, and
z ij m represents the one or more estimated T2 consensus wind estimates.
4. A system comprising:
a plurality of consensus control devices, each comprising at least one processor, that are communicatively coupled to one another, each consensus control device in the plurality of consensus control devices being configured to:
receive, from one or more other consensus control devices in the plurality of consensus control devices, respective T2 consensus wind estimates that each represents a respective estimate of the actual wind speed as determined from the perspective of the respective location associated with each of the one or more other consensus control devices;
determine, based on the respective wind measurement and the respective T2 consensus wind estimates, using an augmented Lagrangian method, a respective T1 consensus wind estimate that represents an estimate of the characteristic of wind as determined from the perspective of the first wind turbine,
receive, from the one or more other consensus control devices, respective T2 comparison factors that each represents a respective amount of offset as determined from the perspective of the respective location associated with each of the one or more other consensus control devices, based on a previous value of the respective T2 comparison factors and a previous value of the respective T2 consensus wind estimates
determine, based on the respective T2 comparison factors, a previous value of the respective T1 consensus wind estimate, and a previous value of respective T1 comparison factors that each represents an amount of offset as determined from the perspective of the respective location with respect to the respective location associated with each of the one or more other consensus control devices, the respective T1 comparison factors; and
determine the respective T1 consensus wind estimate based further on:
the wind measurement,
the previous value of the respective T1 consensus wind estimate
the respective T1 comparison factors
a Lagrangian penalty factor that represents a weighting of the second locations relative to the first location,
a total number of consensus control devices in the plurality of consensus control devices, and
cause a respective wind turbine to adjust an operating parameter based on the respective T1 consensus wind estimate.
5. The system of claim 4 , wherein the augmented Lagrangian methods comprise an alternating direction method of multipliers (ADMM) method.
6. The system of claim 4 , wherein determining the T1 consensus wind estimate comprises determining (x i m+1 ) using
2
x
m
e
a
s
u
r
e
-
ρ
Σ
j
∈
𝒩
(
i
)
(
-
z
ij
m
+
u
ij
m
)
ρ
N
t
u
r
b
s
+
2
wherein:
X measure represents the respective wind measurement,
z ij m represents the respective estimated T2 consensus wind estimates,
u ij m represents the respective T1 comparison factors,
ρ represents the Lagrangian penalty factor, and
N turbs represents the total number of consensus control devices in the plurality of consensus control devices, wherein:
determining, for each respective one of the one or more other consensus control devices, the respective estimated T2 consensus wind estimates z ij m comprises determining
θ
(
x
i
m
+
u
i
j
m
-
1
)
+
(
1
-
θ
)
(
x
j
m
+
u
j
i
m
-
1
)
wherein:
θ represents a scaling factor,
x i m represents the previous value of the respective T1 consensus wind estimate,
u ij m−1 represents a previous value of the respective T1 comparison factors,
x j m represents the respective T2 consensus wind estimates, and
u ji m−1 represents the respective T2 comparison factors; and,
determining, for each respective one of the one or more other consensus control devices, the respective T1 comparison factors (u ij m ) comprises determining u ij m−1 +(x i m −z ij m ) wherein:
u ij m−1 represents the previous value of the respective T1 comparison factors,
x i m represents the previous value of the respective T1 consensus wind estimate, and
z ij m represents the respective estimated T2 consensus wind estimates.
7. A method comprising:
receiving, by a consensus control device comprising at least one processor, a wind measurement that represents a value of a characteristic of wind as measured at a first location;
receiving, by the consensus control device, one or more T2 consensus wind estimates that each represents a respective estimate of the value of the characteristic of wind as determined from the perspective of a respective second location;
determining, by the consensus control device, based on the wind measurement and the one or more T2 consensus wind estimates, using an augmented Lagrangian method, a T1 consensus wind estimate that represents an estimate of the value of the characteristic of wind as determined from the perspective of the first location, and
receiving one or more T2 comparison factors that each represents a respective amount of offset as determined from the perspective of the respective second location, based on a previous value of the one or more T2 comparison factors and a previous value of the one or more T2 consensus wind estimates;
determining, based on the one or more T2 comparison factors, a previous value of the T1 consensus wind estimate, and a previous value of one of one or more T1 comparison factors that each represents an amount of offset as determined from the perspective of the first location with respect to the respective second location, the one or more T1 comparison factors; and
determining the T1 consensus wind estimate based further on:
the wind measurement,
the previous value of the T1 consensus wind estimate,
the one or more T1 comparison factors,
a Lagrangian penalty factor that represents a weighting of the second locations relative to the first location, and
a total number of grouped locations, and
causing, by the consensus control device, a wind turbine to adjust an operating based on the T1 consensus wind estimate.
8. The method of claim 7 , wherein determining the T1 consensus wind estimate comprises determining (x i m+1 ) using
2
x
m
e
a
s
u
r
e
-
ρ
Σ
j
∈
𝒩
(
i
)
(
-
z
i
j
m
+
u
i
j
m
)
ρ
N
t
u
r
b
s
+
2
wherein:
X measure represents the wind measurement,
x ij m represents the one or more estimated T2 consensus wind estimates,
u ij m represents the one or more T1 comparison factors,
ρ represents the Lagrangian penalty factor, and
N turbs represents the total number of grouped locations,
determining, for each respective second location, the one or more estimated T2 consensus wind estimates (z ij m ) comprises determining
θ
(
x
i
m
+
u
i
j
m
-
1
)
+
(
1
-
θ
)
(
x
j
m
+
u
j
i
m
-
1
)
wherein:
θ represents a scaling factor,
x i m represents the previous value of the T1 consensus wind estimate,
u ij m−1 represents a previous value of the one or more T1 comparison factors,
x j m represents the one or more T2 consensus wind estimates, and
u ji m−1 represents the one or more T2 comparison factors; and
determining, for each respective second location, the updated one or more T1 comparison factors (u ij m ) comprises determining u ij m−1 +(x i m −z ij m ) wherein:
u ij m−1 represents the previous value of the one or more T1 comparison,
x i m represents the previous value of the T1 consensus wind estimate, and
z ij m represents the one or more estimated T2 consensus wind estimates.