Multi-scale fusion defogging method based on stacked hourglass network

1. A multi-scale fusion defogging method based on a stacked hourglass network, comprising:

inputting a foggy image into a preset image defogging network; and

outputting a fogless image after the foggy image is processed by the image defogging network;

wherein the image defogging network comprises a 7×7 convolutional layer, a stacked hourglass module, a feature fusion, a multi-scale jump connection module, a 1×1 convolutional layer, a 3×3 convolutional layer, a hierarchical attention distillation module, the 3×3 convolutional layer and the 1×1 convolutional layer connected sequentially;

wherein the stacked hourglass module consists of N fourth-stage hourglass modules in series;

each fourth-stage hourglass module comprises five parallel convolutional streams, wherein an innermost convolutional stream is configured to process an original scale, a second to last convolutional stream and an outermost convolutional stream are configured to downsample to ½, ¼, ⅛ and 1/16, respectively; and

the five parallel convolutional streams are configured to extract features in different resolution groups, and deliver the features of each resolution through a residual module, to be recovered to the original scale through an up sample layer and be fused after recovery;

wherein the fourth-stage hourglass module is formed by replacing a residual module at a middle of a fourth row of a third-stage hourglass module with a first-stage hourglass module;

the third-stage hourglass module is formed by replacing a residual module at a middle of a third row of a second-stage hourglass module with the first-stage hourglass module;

the second-stage hourglass module is formed by replacing a residual module at a middle of a second row of the first-stage hourglass module with the first-stage hourglass module; and

the first-stage hourglass module comprises a first row comprising a residual module and a second row comprising a max pool layer, three residual modules and the up sample layer in sequence, wherein the first row and the second row of the first-stage hourglass module are configured to fuse and output the features;

wherein each residual module consists of a first row being a skip level layer comprising the 1×1 convolutional layer, and a second row being a convolutional layer that comprises a batch normalization (BN) layer, a rectified linear unit (Relu) layer, the 1×1 convolutional layer, the BN layer, the Relu layer, the 3×3 convolutional layer, the BN layer, the Relu layer and the 1×1 convolutional layer; and

fusing and outputting the features at outputs of the skip level layer and the convolutional layer.

2. The multi-scale fusion defogging method according to claim 1 , wherein the N is 8.

3. A multi-scale fusion defogging method based on a stacked hourglass network, comprising:

inputting a foggy image into a preset image defogging network; and

outputting a fogless image after the foggy image is processed by the image defogging network;

wherein the image defogging network comprises a 7×7 convolutional layer, a stacked hourglass module, a feature fusion, a multi-scale jump connection module, a 1×1 convolutional layer, a 3×3 convolutional layer, a hierarchical attention distillation module, the 3×3 convolutional layer and the 1×1 convolutional layer connected sequentially;

the multi-scale jump connection module comprises a first row consisting of three 3×3 convolutional layers and a Relu layer in series, a second row consisting of three 5×5 convolutional layers and the Relu layer in series, and a third row consisting of three 7×7 convolutional layers and the Relu layer in series;

taking outputs of a first 3×3 convolutional layer and the Relu layer of each row as inputs of a second 3×3 convolutional layer and the Relu layer of each row, respectively;

taking outputs of the second 3×3 convolutional layer and the Relu layer of each row as inputs of a third 3×3 convolutional layer and the Relu layer of each row, respectively; and

fusing outputs of the third 3×3 convolutional layer and the Relu layer of each row through a contact module and outputting after fusion.

4. A multi-scale fusion defogging method based on a stacked hourglass network, comprising:

inputting a foggy image into a preset image defogging network; and

outputting a fogless image after the foggy image is processed by the image defogging network;

wherein the image defogging network comprises a 7×7 convolutional layer, a stacked hourglass module, a feature fusion, a multi-scale jump connection module, a 1×1 convolutional layer, a 3×3 convolutional layer, a hierarchical attention distillation module, the 3×3 convolutional layer and the 1×1 convolutional layer connected sequentially;

wherein the hierarchical attention distillation module comprises a channel attention module and a spatial attention module, and

fusing outputs of the channel attention module and the spatial attention module and outputting after fusion;

processing, by the channel attention module, an input feature map F through a global max pool layer in H dimension and a global avgpool layer in W dimension respectively, to obtain two 1×1×C feature maps, wherein the input feature map F is expressed by a formula F=H×W×C, H denotes height, W denotes width, C denotes a number of channels; and

inputting the two 1×1×C feature maps into a two-layer neural network with shared weights for learning inter-channel dependencies; and

summing and fusing features output from a multilayer perceptron (MLP); and

operating by a sigmoid function after fusion to generate a weight M of channels.

5. The multi-scale fusion defogging method according to claim 4 , further comprising

processing, by the spatial attention module, an input feature map F through a max pool layer in C dimension and an avgpool layer in C dimension respectively, to obtain two H×W×1 feature maps, wherein the input feature map F is expressed by a formula F=H×W×C, H denotes height, W denotes width, C denotes a number of channels;

splicing the two H×W×1 feature maps based on a channel dimension;

reducing the channel dimension on the spliced feature map by using the 7×7 convolution layer; and

operating by a sigmoid function after fusion to generate a weight M of a spatial dimension.