Hyperspectral image classification method based on context-rich networks

1. A hyperspectral image classification method based on context-rich networks, comprising following steps:

step 1, pre-processing a hyperspectral image;

step 2, selecting a training set, and generating a ground truth label map with the same width and height as the image in step 1, with a value of a category ID, and setting pixels that are not position selected in the label map as a background value to be ignored in subsequent calculation of a loss function;

step 3, constructing a context-rich network, an overall structure of the network is divided into three parts: a feature extraction module, a context-rich information capturing module, and a classification module;

wherein the feature extraction module is used to process the inputted pre-processed hyperspectral image to obtain a feature X;

a specific process of the context-rich information capturing module is: the feature X obtained by the feature extraction module is meshed into grids of different sizes in different paths parallelly, in each path, an intra-feature relationship extraction is performed within each grid in the grids respectively to complete a spatial context information aggregation, a PAM module (parallel attention module) is used to realize a spatial context information extraction process in the grids; a feature set is obtained after completing the spatial context information extraction process in each path, then, a scale context-aware module is used to extract a contextual relationship between features, obtaining the features that have both spatial and scale contexts, finally, inputting this feature into the classification module;

the classification module is used to predict a classification map;

step 4, training the context-rich network with the training set to achieve convergence of the loss function;

step 5, a hyperspectral image classification is completed by inputting the image pre-processed and to be classified in step 1 to the trained context-rich network.

2. The hyperspectral image classification method based on context-rich networks according to claim 1 , wherein: the pre-processing is a Z-score standardization or a normalization process on the hyperspectral image, as following implementations:

performing the normalization process on hyperspectral data X 0 through the formula

x

ijc

′

=

2

⁢

(

x

ijc

-

x

cmin

)

x

cmax

-

x

cmin

-

1

⁢

or

⁢

x

ijc

′

=

x

ijc

-

μ

c

σ

c

,

wherein x ijc denotes a value of a pixel x ij at a band c in a row i and a column j of the X 0 , x cmin , x cmax are a minimum value and a maximum value of the band c in the X 0 , μ c and σ c are a mean and a standard deviation of the band c, the generated data is represented by X′ 0 .

3. The hyperspectral image classification method based on context-rich networks according to claim 1 , wherein: the feature extraction is directly passing the inputted pre-processed hyperspectral image through five convolutional layers, these convolutional layers all contain three operations including a convolution, a group normalization, and a nonlinear mapping, a numbers of convolution kernels of the five convolutional layers are 64, 112, 160, 208 and 256, respectively, wherein a first convolutional layer is followed by an average pooling layer.

4. The hyperspectral image classification method based on context-rich networks according to claim 1 , wherein: the classification module includes five operations: a first convolution, a group normalization, a nonlinear mapping, a second convolution, and a softmax function, wherein the first convolution contains 128 convolution kernels, and a number of convolution kernels in the second convolution is the same as a number of categories.

5. The hyperspectral image classification method based on context-rich networks according to claim 1 , wherein: during the training process, network parameters adopt Xavier initialization, and when optimizing the parameters, a stochastic gradient descent algorithm with momentum is used to minimize a loss function of the network, a loss function is a cross-entropy loss function, the trained network can effectively classify context-rich features of the hyperspectral image.

6. The hyperspectral image classification method based on context-rich networks according to claim 1 , wherein:

in the context-rich information capturing module, before performing intra-feature context information extraction, a dimension of the feature X={X i ∈ C 0 ×H×W } i=1 N s obtained by the feature extraction module is reduced with a 1×1 convolution, wherein C 0 , H and W are a channel number, a width and a height of X, respectively, and N s is a number of paths; in each path, the intra-feature relationship extraction is respectively performed in each grid to complete the spatial context information aggregation; for a path i that has n×n grids, that is, divided into n parts in a width direction and a height direction, respectively, wherein size of each grid is

⌊

H

n

⌋

×

⌊

W

n

⌋

,

firstly, three 1×1 convolutions are respectively used to map the dimension-reduced X to three features: Q i , K i , V i , for a grid g, the features within a coverage of this grid are taken respectively from features Q i , K i , V i and perform matrix shapes transformation, wherein both a channel number of Q i , K i are c1, and the channel number of V i is c2, obtaining q g ∈ c1×hw , k g ∈ c1×hw , v g ∈ c2×hw ; using the q g , k g , v g respectively as a query feature, a key feature and a value feature for realizing the spatial context information extraction process in the grid, and generating a new feature p g ∈ c2×h×w through the matrix shape transformation, wherein

h

=

⌊

H

n

⌋

,

w

=

⌊

W

n

⌋

,

this process can be formulated as

p g =softmax( q g T k g ) v g T

then, p g is filled to a position covered by grid g in the dimension-reduced feature X i , while all the other grids are processed in same way, thus obtaining a new feature P i that completes spatial contexts in that path, and each path will generate a new feature, denoted as a feature set P={P i ∈ C 1 ×H×W } i=1 N s , wherein C 1 is a number of channels, C 1 =c2=4c1.

7. The hyperspectral image classification method based on context-rich networks according to claim 6 , wherein: a specific implementation process of extracting the inter-feature relationship using the scale context-aware module is as follows;

firstly, three groups of 1×1 convolution are utilized to respectively map feature P and concatenate results in a scale dimension, obtaining the query feature Q∈ N s ×C 2 ×H×W , the key feature K∈ N s ×C 2 ×H×W and the value feature V∈ N s ×C 1 ×H×W , wherein

C

2

=

C

1

4

,

then, performing the matrix shapes transformation on Q, K, V to obtain Q′∈ N s ×C 2 HW , K′∈ N s ×C 2 HW and V′∈ N s ×C 1 HW , and a scale attention map M∈ N s N s is obtained through below formula:

M

ij

=

Q

i

′

⁢

K

j

′

∑

j

=

1

N

s

⁢

Q

i

′

⁢

K

j

′

wherein i and j represent two different scales respectively, that is, the indexes of two paths, and also the row and column positions of a corresponding attention value in the scale attention map; multiplying the scale attention map M with the value feature V′ to obtain a new feature S that simultaneously perceive intra-feature and inter-feature context dependencies:

S=MV′

finally, after concatenating S and X in a channel dimension, inputting it into the classification module.