Tensor decomposition-based big data processing system and processing method for spectrum monitoring
A method of processing spectrum monitoring (SM) big data based on tensor decomposition comprises the steps of: S 1 : processing calibration of geolocation V, synchronized clock t, synchronized time t n (0 . . . N); of SM stations and determining SM sampling point M and bandwidth B; S 2 : processing discretization for a monitoring time and structured processing of SM data for a monitoring period to obtain a one-dimensional SM sequence I t n at the given sampling time and a two-dimensional SM matrix W at the given monitoring period; S 3 : constructing a cuboid matrix Q based on the two-dimensional SM matrix W, processing tensor decomposition for the cuboid matrix Q and identify the emitter.
1. A method of processing big data for regional spectrum monitoring based on tensor decomposition, comprising the step of:
S 1 : processing calibration of parameters of a spectrum monitoring station, and determining spectrum monitoring sampling point M and bandwidth B, wherein the parameters comprise a geolocation V, a synchronized clock t, a synchronized time t n (0 . . . N);
S 2 : processing discretization of spectrum monitoring data at a sampling time completely and a structured processing of spectrum monitoring data for a monitoring period to obtain a one-dimensional spectrum monitoring sequence I t n at the given sampling time and a two-dimensional spectrum monitoring matrix W at the given monitoring period; and
S 3 : constructing a cuboid matrix Q based on the two-dimensional spectrum monitoring matrix W and processing tensor decomposition for the cuboid matrix Q,
wherein step S 3 comprises the steps of:
S 3 . 1 : calculating a geographical center point V 0 on a geographical distribution according to latitude and longitude positions V n of K number of spectrum monitoring stations, where n equals to 1, 2, 3, . . . , K, where
V
0
=
1
K
∑
i
=
1
K
V
i
S 3 . 2 : calculating the distance D n between V n of each spectrum monitoring station and the geographic center point V 0 , where ∥⋅∥ 2 is the second-order norm:
D n =∥V n −V 0 ∥ 2
S 3 . 3 : along an ascending order of the distance D n between the spectrum monitoring station and the geographic center point V 0 , arranging the corresponding spectrum matrix of the station to construct the spectrum monitoring network cuboid matrix Q, where Q=[W 1 , W 2 , . . . W K ] T ;
S 3 . 4 : performing tensor decomposition on the three-dimensional cuboid matrix Q, setting an analysis bandwidth F and dividing the cuboid matrix Q according to the analysis bandwidth F into segments, processing tensor decomposition for each segment, determining a validity of the tensor decomposition by setting MF>80% as a criterion of the validity; if MF<80%, the analysis bandwidth F is reduced to half, and the segments that does not meet the validity requirement is divided into segments according to the reduced analysis bandwidth F to re-process with tensor decomposition until the validity requirement are met or the minimum analysis bandwidth is reached;
S 3 . 5 : based on the results of tensor decomposition, calculating a center frequency, frequency occupancy, time occupancy, space occupancy of the emitter, or evaluating the environmental complexity;
S 3 . 6 : performing K-L divergence and mutual information calculations on the co-frequency emitters to identify whether it is an independent emitter or not.
2. The method according to claim 1 , wherein step S 3 . 5 comprises the steps of:
S 3 . 5 . 1 : Obtaining loading matrices A, B, C after performing tensor decomposition, where Y (1) =(A⊙B)C T , Y (1) ∈C IJ×K ; Y (2) =(C⊙A)B T , Y (2) ∈C KI×J , Y (3) =(B⊙C)A T , Y (3) ∈C JK×I , where ⊙ is Khatri-Rao product, I, J, K are frequency, time and space (station) points respectively;
S 3 . 5 . 2 : converting Y (1) into a cuboid matrix Y (1) ∈C I×J×K , where there are a total of K time-frequency slices along the spatial dimension, after adding up along the time dimension, dividing the number of peaks exceeding the threshold by F to obtain a frequency occupancy FO within the analysis bandwidth; after adding up along the time dimension, dividing the number of peaks exceeding the threshold by the time length J to obtain a time occupancy TO during the monitoring period;
S 3 . 5 . 3 : converting Y (2) into a cuboid matrix Y (2) ∈C K×I×J , Y (3) into a cuboid matrix Y (3) ∈C K×I×J , after adding up along the time dimension and the frequency dimension, dividing the number of peaks exceeding the threshold by K to obtain a space occupancy SO;
S 3 . 5 . 4 : evaluating the environmental complexity and classifying the environmental complexity as simple, mild, moderate, and severe based on the following standards:
Simple
electromagnetic
environment
:
0
≤
FO
×
TO
×
SO
3
≤
10
%
,
Mild
electromagnetic
environment
:
10
%
<
FO
×
TO
×
SO
3
≤
40
%
,
Moderate
electromagnetic
environment
:
40
%
<
FO
×
TO
×
SO
3
≤
70
%
,
Severe
electromagnetic
environment
:
70
%
<
FO
×
TO
×
SO
3
≤
100
%
.
3. The method according to claim 1 , further comprising the steps of:
S 4 : performing data transmission, transmitting emitter identification result and compressed data to a management center, a data center or a relay station;
S 5 : under control of the management center, calculating three-dimensional coordinates of the emitter by using geolocation information of grid node and time difference between the grid node and the reference node receiving the target emitter signal.
4. A method of processing big data for regional spectrum monitoring based on tensor decomposition, comprising the step of:
S 1 : processing calibration of parameters of a spectrum monitoring station, and determining spectrum monitoring sampling point M and bandwidth B, wherein the parameters comprise a geolocation V, a synchronized clock t, a synchronized time t n (0 . . . N);
S 2 : processing discretization of spectrum monitoring data at a sampling time completely and a structured processing of spectrum monitoring data for a monitoring period to obtain a one-dimensional spectrum monitoring sequence I t n at the given sampling time and a two-dimensional spectrum monitoring matrix W at the given monitoring period; and
S 3 : constructing a cuboid matrix Q based on the two-dimensional spectrum monitoring matrix W and processing tensor decomposition for the cuboid matrix Q,
S 4 : performing data transmission, transmitting emitter identification result and compressed data to a management center, a data center or a relay station; and
S 5 : under control of the management center, calculating three-dimensional coordinates of the emitter by using geolocation information of grid node and time difference between the grid node and the reference node receiving the target emitter signal,
wherein step S 5 comprises the steps of:
S 5 . 1 : listing emitter characteristic parameters sent by the management center to the gateway nodes of each sub-network, sorting monitoring stations according to signal strength of a target emitter received by the monitoring station, and determining a number of stations that effectively receive an emitter signal N;
S 5 . 2 : forming an emitter positioning grid by the monitoring stations and sorting the monitoring stations according to S 3 . 1 ˜S 3 . 3 to determine the reference station if the number of monitoring stations N≥4;
S 5 . 3 : under the control of the management center, measuring an arrival time of the target emitter signal of each of the monitoring stations based on a calculation of a distance between the particular monitoring station and the reference station, and sending results of measurement to the management center;
S 5 . 4 : the management center estimating geolocation of the emitter based on a difference between the arrival time of the target emitter signal and the distance between the particular monitoring station and the reference station.
5. The method of processing big data for regional spectrum monitoring based on tensor decomposition according to claim 4 , wherein step S 2 comprises the steps of:
S 2 . 1 : obtaining the spectrum monitoring data at the sampling time;
S 2 . 2 : processing discretization of the spectrum monitoring data at the sampling time within a given bandwidth B based on a required number of sampling points M to obtain the one-dimensional spectrum monitoring sequence I t n at the given sampling time, and a dimension of I t n is 1×M;
S 2 . 3 : obtaining the spectrum monitoring sequence I t 0 I t 1 . . . I t N at different sampling time for a given monitoring period according to a sampling time interval of the monitoring data;
S 2 . 4 : aligning the spectrum monitoring sequence at different sampling time in chronological order to form a two-dimensional matrix W=[I t 0 I t 1 . . . I t N ] T , the matrix W is a two-dimensional matrix in N×M dimension.
6. A spectrum monitoring big data processing system based on tensor decomposition, which comprises a station calibration and synchronization module, a spectrum monitoring data preprocessing module, and a spectrum monitoring big data processing module,
wherein the station calibration and synchronization module processes calibration of parameters of a spectrum monitoring station, and determining spectrum monitoring sampling point M and bandwidth B, wherein the parameters comprise a geolocation V, a synchronized clock t, asynchronized time t n (0 . . . N);
the spectrum monitoring data preprocessing module completes discretization processing of spectrum monitoring data at a given monitoring time and structured processing of spectrum monitoring data for a given monitoring period to obtain a one-dimensional spectrum monitoring sequence I t n at the given monitoring time and a two-dimensional spectrum monitoring matrix W at the given monitoring period;
the spectrum monitoring big data processing module constructs a cuboid matrix Q based on the two-dimensional spectrum monitoring matrix W and processes tensor decomposition for the cuboid matrix Q,
wherein the spectrum monitoring data preprocessing module comprises:
a sampling time data discretization unit which completes the discretization of monitoring data at the given sampling time according to a required number of spectrum sampling points within the given spectrum bandwidth; and
a monitoring data structure processing unit which obtains data of each of the monitoring stations according to the time sampling interval of the monitoring data within the given monitoring time period, and arranges the spectrum monitoring sequence at different sampling time moment in chronological order to form a two-dimensional spectrum matrix,
wherein the spectrum monitoring big data processing module processes the steps of:
S 1 : calculating a geographical center point V 0 on a geographical distribution according to latitude and longitude positions V n of K number of spectrum monitoring stations, where n equals to 1, 2, 3, . . . , K, where
V
0
=
1
K
∑
i
=
1
K
V
i
S 2 : calculating the distance D n between V n of each spectrum monitoring station and the geographic center point V 0 , where ∥⋅∥ 2 is the second-order norm:
D n =∥V n −V 0 ∥ 2
S 3 : along an ascending order of the distance D n between the spectrum monitoring station and the geographic center point V 0 , arranging the corresponding spectrum matrix of the station to construct the spectrum monitoring network cuboid matrix Q, where Q=[W 1 , W 2 , . . . W K ] T ,
S 4 : performing tensor decomposition on the three-dimensional cuboid matrix Q, setting an analysis bandwidth F and dividing the cuboid matrix according to the analysis bandwidth F into segments, processing tensor decomposition for each segment, determining a validity of the tensor decomposition by setting model fitness MF>80% as a criterion of the validity; if MF<80%, the analysis bandwidth F is reduced to half, and the segments that does not meet the validity requirement is divided into segments according to the reduced analysis bandwidth F to re-process with tensor decomposition until the validity requirement are met or the minimum analysis bandwidth is reached;
S 5 : based on the results of tensor decomposition, calculating a center frequency, frequency occupancy, time occupancy, space occupancy of the emitter, or evaluating the environmental complexity;
S 6 : performing K-L divergence and mutual information calculations on the co-frequency emitters to identify whether it is an independent emitter or not.
7. The spectrum monitoring big data processing system based on tensor decomposition according to claim 6 , wherein the station calibration and synchronization module comprises:
a geolocation calibration unit which obtains accurate geolocation information of the spectrum monitoring station;
a monitoring time calibration unit which calibrates the clock of the monitoring station; and
a parameter synchronization unit which synchronously acquires initial spectrum monitoring parameters, which include: a sampling time period, a monitoring data sampling interval, a monitoring frequency band, and a monitoring spectrum sampling rate.