FEATURE FUSION FOR INPUT PICTURE DATA PREPROCESSING FOR LEARNING MODEL
Methods and systems implement input picture data preprocessing for a learning model by picture data blurring based on deep features. Intermediate features are extracted from convolutional layers of a preprocessing model, and each set of intermediate features are fused to yield a fused feature map, and enlarged to input picture size. Based on the fused feature map, the preprocessing model can configure one or more processors of an input preprocessing computing system to, in performing blurring preprocessing computations, emphasize picture data having larger corresponding characteristic values, and deemphasize other picture data.
1 . A method comprising:
performing, by one or more processors of an input preprocessing computing system, a plurality of convolutions upon input picture data;
outputting, by the one or more processors, a plurality of intermediate feature maps from respective different convolutions;
averaging, by the one or more processors, absolute values of the plurality of intermediate feature maps to yield a fused feature map; and
resizing the fused feature map to a size of the input picture data.
2 . The method of claim 1 , further comprising performing, by the one or more processors, a Gaussian blurring transformation upon a pixel of the input picture data, the Gaussian blurring transformation taking as input a feature map value corresponding to the pixel from the fused feature map.
3 . The method of claim 2 , wherein the feature map value corresponding to the pixel is computed, by the one or more processors, as a standard deviation value in the Gaussian blur transformation.
4 . The method of claim 2 , wherein the one or more processors perform a Gaussian blurring transformation upon each pixel of the input picture data.
5 . The method of claim 2 , wherein the plurality of convolutions are performed by the one or more processors during segmentation computations performed upon the input picture data to output an object mask.
6 . The method of claim 5 , further comprising modifying, by the one or more processors, the object mask to exclude each pixel of a sliding window.
7 . The method of claim 6 , further comprising multiplying, by the one or more processors, the modified object mask and the input picture data to output block-based masked input picture data.
8 . The method of claim 7 , wherein the one or more processors perform a Gaussian blurring transformation upon the block-based masked input picture data.
9 . The method of claim 8 , further comprising deciding, by the one or more processors, based on average object mask ratio of a video sequence exceeding a threshold, to perform a Gaussian blurring transformation upon the block-based masked input picture data.
10 . The method of claim 8 , further comprising deciding, by the one or more processors, based on temporal complexity of a video sequence of a video sequence exceeding a threshold, to perform a Gaussian blurring transformation upon the block-based masked input picture data.
11 . A computing system comprising:
one or more processors, and
a computer-readable storage medium communicatively coupled to the one or more processors, the computer-readable storage medium storing computer-readable instructions executable by the one or more processors that, when executed by the one or more processors, perform associated operations comprising:
performing a plurality of convolutions upon input picture data;
outputting a plurality of intermediate feature maps from respective different convolutions;
averaging absolute values of the plurality of intermediate feature maps to yield a fused feature map; and
resizing the fused feature map to a size of the input picture data.
12 . The computing system of claim 11 , wherein the one or more processors are further configured to blur input picture data by performing a Gaussian blurring transformation upon a pixel of the input picture data, the Gaussian blurring transformation taking as input a feature map value corresponding to the pixel from the fused feature map.
13 . The computing system of claim 12 , wherein the one or more processors are configured to compute the feature map value corresponding to the pixel as a standard deviation value in the Gaussian blur transformation.
14 . The computing system of claim 12 , wherein the one or more processors are configured to perform a Gaussian blurring transformation upon each pixel of the input picture data.
15 . The computing system of claim 12 , wherein the one or more processors are configured to perform the plurality of convolutions during segmentation computations performed upon the input picture data to output an object mask.
16 . The computing system of claim 15 , wherein the one or more processors are further configured to modify the object mask to exclude each pixel of a sliding window.
17 . The computing system of claim 16 , wherein the one or more processors are further configured to multiply the modified object mask and the input picture data to output block-based masked input picture data.
18 . The computing system of claim 17 , wherein the one or more processors are configured to perform a Gaussian blurring transformation upon the block-based masked input picture data.
19 . The computing system of claim 18 , wherein the one or more processors are further configured to decide, based on average object mask ratio of a video sequence exceeding a threshold, to perform a Gaussian blurring transformation upon the block-based masked input picture data.
20 . The computing system of claim 18 , wherein the one or more processors are further configured to decide, based on temporal complexity of a video sequence of a video sequence exceeding a threshold, to perform a Gaussian blurring transformation upon the block-based masked input picture data.