Joint blur map estimation and blur desirability classification from an image
Techniques are disclosed for blur classification. The techniques utilize an image content feature map, a blur map, and an attention map, thereby combining low-level blur estimation with a high-level understanding of important image content in order to perform blur classification. The techniques allow for programmatically determining if blur exists in an image, and determining what type of blur it is (e.g., high blur, low blur, middle or neutral blur, or no blur). According to one example embodiment, if blur is detected, an estimate of spatially-varying blur amounts is performed and blur desirability is categorized in terms of image quality.
1. A method for classifying blur in an image comprising:
processing an image to generate a blur map (B m ), a content feature map (C m ), and an attention map (A m );
combining said blur map, said content feature map, and said attention map to generate a composite map; and,
performing classification on said composite map to generate a blur type classification.
2. The method according to claim 1 , wherein said composite map comprises a stack of B m ×A m , B m ×(1−A m ) and C m .
3. The method according to claim 1 , wherein said blur type classification is one of the classifications GOOD BLUR, OK BLUR, BAD BLUR, and NO BLUR.
4. The method according to claim 1 , wherein said content feature map is generated using a ResNet model.
5. The method according to claim 1 , wherein said blur map is generated by performing a dilated convolution operation, a pyramid pooling operation, a bilinear up-sampling operation, and a dense connected boundary refinement operation.
6. The method according to claim 1 , wherein said attention map is generated by performing a dilated convolution operation, a pyramid pooling operation, a bilinear up-sampling operation, and a dense connected boundary refinement operation.
7. The method according to claim 6 , further comprising performing a joint training of a blur map estimator, an attention estimator block, and a content feature map estimator block.
8. A system for classifying blur in an image, the system comprising:
a dilated convolution block, said dilated convolution block to perform a dilated convolution on an image;
a blur map estimator block coupled to an output of said dilated convolution block, said blur map estimator block to generate a blur map;
an attention estimator block coupled to an output of said dilated convolution block, said attention map estimator block to generate an attention map;
a content feature map estimator block, said content feature map estimator block to generate a content feature map based upon said image;
a combiner, said combiner to generate a composite map from said blur map, said attention map, and said content feature map; and,
a blur category classifier coupled to an output of said combiner, said blur category classifier to generate a blur type classification based upon said composite map.
9. The system according to claim 8 , wherein said composite map comprises a stack of B m ×A m , B m ×(1−A m ) and C m .
10. The system according to claim 8 , wherein said blur type classification is one of GOOD BLUR, OK BLUR, BAD BLUR, and NO BLUR.
11. The system according to claim 8 , wherein said content feature map estimator block is a ResNet model.
12. The system according to claim 8 , wherein each of said blur map estimator and said attention map estimator comprises a dilated convolution block, a pyramid pooling block, a bilinear up-sampling block, and a dense connected boundary refinement block.
13. The system according to claim 8 , further comprising a training block to jointly train said blur map estimator, said attention estimator block, and said content feature map estimator block.
14. A computer program product including one or more non-transitory machine-readable mediums encoded with instructions that when executed by one or more processors cause a process to be carried out for classifying blur in an image, the process comprising:
processing an image to generate a blur map (B m ), a content feature map (C m ), and an attention map (A m );
combining said blur map, said content feature map, and said attention map to generate a composite map; and,
performing classification on said composite map to generate a blur type classification.
15. The computer program product according to claim 14 , wherein said composite map comprises a stack of B m ×A m , B m ×(1−A m ) and C m .
16. The computer program product according to claim 14 , wherein said blur type classification is one of the classifications GOOD BLUR, OK BLUR, BAD BLUR, and NO BLUR.
17. The computer program product according to claim 14 , wherein said content feature map is generated using a ResNet model.
18. The computer program product according to claim 14 , wherein said blur map is generated by performing a dilated convolution operation, a pyramid pooling operation, a bilinear up-sampling operation, and a dense connected boundary refinement operation.
19. The computer program product according to claim 14 , wherein said attention map is generated by performing a dilated convolution operation, a pyramid pooling operation, a bilinear up-sampling operation and a dense connected boundary refinement operation.
20. The computer program product according to claim 14 , the process further comprising jointly training a blur map estimator, an attention estimator block, and a content feature map estimator block.