Short-term irradiance prediction method based on image processing and multimodal data fusion, device thereof and storage medium
Disclosed are a short-term irradiance prediction method based on image processing and multimodal data fusion, a device thereof and a storage medium in the field of solar irradiance prediction. The method includes the following steps: acquiring irradiance data, and carrying out blurring processing on the irradiance data, where the irradiance data which has been subjected to blurring processing includes time information and irradiance information; converting the time information into an image layer containing the position of the sun, and drawing a circular area with the solar imaging center coordinate as the center and the radius r to obtain a time information layer image; fusing a normally exposed all-sky image, an underexposed all-sky image and a time information layer image, inputting the fused image data into a pre-built short-term solar irradiance prediction model, outputting a short-term irradiance prediction value.
1 . A short-term irradiance prediction method based on image processing and multimodal data fusion, comprising the following steps:
acquiring irradiance data, and carrying out blurring processing on the irradiance data to obtain irradiance data which has been subjected to blurring processing, wherein the irradiance data which has been subjected to blurring processing comprises time information and irradiance information;
calculating a solar imaging center coordinate based on an astronomical principle and a fish-eye camera image principle, and drawing a circular area with the solar imaging center coordinate as the center and the radius r to generate a time information layer image;
fusing the time information layer image with a normally exposed all-sky image and an underexposed all-sky image which are acquired in advance to obtain fused image data; and
inputting the fused image data into a pre-built short-term solar irradiance prediction model, and outputting a short-term irradiance prediction value; wherein the short-term solar irradiance prediction model comprises a patch division layer, a linear embedding layer, a Swin-Transformer block and a patch merging layer, and the step of outputting a short-term irradiance prediction value comprises:
S 1 , dividing the fused image data into non-overlapping patch vectors through the patch division layer;
S 2 , mapping a plurality of patch vectors to high dimensions through the linear embedding layer to extract complex features;
S 3 , calculating the attention between pixels in the patch by using the patch vector through the Swin-Transformer block to establish a relationship between different patches;
S 4 , alternately stacking the different patches through the patch merging layer and the Swin-Transformer block for multiple times to extract depth features of the fused image data; and
S 5 , fusing the depth features by using a multilayer perceptron, and processing the fused depth features by using a Softmax activation function to obtain a short-term irradiance prediction value; and
carrying out deblurring processing on the short-term irradiance prediction value to obtain an actual short-term irradiance prediction value; wherein taking the irradiance data which has been subjected to blurring processing as the target data, the similarity between the actual short-term irradiance prediction value and the target data is calculated by using a KullbackLeibler loss function.
2 . The short-term irradiance prediction method based on image processing and multimodal data fusion according to claim 1 , wherein the calculation expression of carrying out blurring processing on the irradiance data is as follows:
y
=
1
2
π
σ
e
-
(
v
-
y
m
)
2
σ
2
;
where y m denotes an average global horizontal irradiance value within a predicted time scale; y denotes the result of blurring of the average global horizontal irradiance; σ denotes a Gaussian blur variance; v denotes an irradiance data set, and π denotes the ratio of the circumference of a circle to its diameter.
3 . The short-term irradiance prediction method based on image processing and multimodal data fusion according to claim 1 , wherein generating a time information layer image comprises:
calculating a zenith angle and a solar azimuth angle at local shooting time according to the astronomical principle, and calculating a solar imaging radius according to the fish-eye camera;
using the solar azimuth angle to calculate the solar imaging center coordinate at the local shooting time based on the solar imaging radius; and
obtaining the time information layer image by drawing a circular area with the solar imaging center coordinate as the center and the radius r on a background image layer with a preset size.
4 . The short-term irradiance prediction method based on image processing and multimodal data fusion according to claim 3 , wherein the expression of calculating a zenith angle and a solar azimuth angle at local shooting time is as follows:
h
a
=
15
×
(
t
+
(
L
-
L
h
)
-
12
)
;
δ
=
-
23.44
°
×
cos
(
2
π
365
(
N
′
+
10
)
)
;
θ
s
=
90
°
-
arcsin
(
cosh
a
cos
δcosϕ
+
sin
δsinϕ
)
;
Ψ
=
arccos
(
sin
δ
-
cos
θ
s
sin
ϕ
sin
θ
s
cos
ϕ
)
;
where h a denotes a solar hour angle at the shooting time; t denotes the time when the all-sky image is taken; L denotes the longitude of the shooting location; L h denotes a time zone center line at the shooting location; & denotes a declination at the shooting time; N′ denotes the number of days in a year; θ s denotes a zenith angle at the local shooting time; Ψ denotes a solar azimuth angle at the shooting time; φ denotes the longitude and the latitude of the time zone center line.
5 . The short-term irradiance prediction method based on image processing and multimodal data fusion according to claim 3 , wherein calculating a solar imaging radius according to the fish-eye camera comprises:
using a nonlinear fitting method to perform nonlinear correction on the imaging angle for imaging positioning, in which the nonlinear correction has a formula of:
θ
=
log
(
180
-
θ
s
)
log
90
×
θ
s
;
where θ denotes a viewing angle after the nonlinear correction;
calculating the solar imaging radius based on the determined imaging positioning, in which the expression of calculating the solar imaging radius is as follows:
l
=
(
θ
90
×
R
)
;
where l denotes a solar imaging radius; R denotes a pixel radius of the all-sky image.
6 . The short-term irradiance prediction method based on image processing and multimodal data fusion according to claim 1 , wherein the expression of calculating the KullbackLeibler loss function is as follows:
J
(
W
)
=
∑
i
N
∑
j
M
y
ij
log
(
y
ij
y
^
ij
)
;
where J(W) denotes a loss value of the prediction model; N denotes the number of samples in a data set; M denotes the number of elements in each sample in the data set; i denotes a sample index; j denotes an element index in each sample; y ij denotes a j-th element of an i-th sample; ŷ ij denotes a j-th element of an i-th value output by the short-term irradiance prediction model.
7 . A non-transitory computer-readable storage medium on which a computer program is stored, wherein the program, when executed by a processor, implements the steps of the method according to claim 1 .
8 . A short-term irradiance prediction device based on image processing and multimodal data fusion, comprising:
a blurring processing module, which is configured to acquire irradiance data, and carry out blurring processing on the irradiance data to obtain irradiance data which has been subjected to blurring processing, wherein the irradiance data which has been subjected to blurring processing comprises time information and irradiance information;
a time information layer image generation module, which is configured to convert the time information into an image layer containing the position of the sun based on an astronomical principle and a fish-eye camera image principle, and draw a time information layer image with the solar imaging center coordinate as the center and the radius r;
an image fusion module, which is configured to fuse a normally exposed all-sky image, an underexposed all-sky image and the time information layer image to obtain fused image data; and
a prediction module, which is configured to input the fused image data into a pre-built short-term solar irradiance prediction model, and output a short-term irradiance prediction value; wherein the short-term solar irradiance prediction model comprises a patch division layer, a linear embedding layer, a Swin-Transformer block and a patch merging layer, and the step of outputting a short-term irradiance prediction value comprises:
S 1 , dividing the fused image data into non-overlapping patch vectors through the patch division layer;
S 2 , mapping a plurality of patch vectors to high dimensions through the linear embedding layer to extract complex features;
S 3 , calculating the attention between pixels in the patch by using the patch vector through the Swin-Transformer block to establish a relationship between different patches;
S 4 , alternately stacking the different patches through the patch merging layer and the Swin-Transformer block for multiple times to extract depth features of the fused image data; and
S 5 , fusing the depth features by using a multilayer perceptron, and processing the fused depth features by using a Softmax activation function to obtain a short-term irradiance prediction value; and
carry out deblurring processing on the short-term irradiance prediction value to obtain an actual short-term irradiance prediction value; wherein taking the irradiance data which has been subjected to blurring processing as the target data, the similarity between the short-term irradiance prediction value and the target data is calculated by using a KullbackLeibler loss function.