Image processing method and apparatus, and storage medium
View Patent ↗A method includes: processing a captured original image based on an image signal processor (ISP) parameter, to obtain a target image; invoking, based on the target image, a pre-trained image quality assessment model to output and obtain a target assessment result, where the target assessment result indicates image quality of the assessed target image; and adjusting the ISP parameter based on the target assessment result.
1 . A method implemented by a processor, wherein the method comprises: training, by using a sample image, a neural network to obtain a pre-trained image quality assessment model, wherein the pre-trained image quality assessment model comprises a scene classification layer, an objective scoring layer, and a subjective scoring layer; processing, based on an image signal processor (ISP) parameter, a captured original image, to obtain a target image; invoking, based on the target image, the pre-trained image quality assessment model to obtain a target assessment result, wherein the target assessment result indicates image quality of the target image, and wherein invoking the pre-trained image quality assessment model comprises: inputting the target image to the scene classification layer to obtain a scene classification result; inputting the target image and the scene classification result to the objective scoring layer to obtain objective quality information; and inputting the target image and the scene classification result to the subjective scoring layer to obtain subjective quality information; and adjusting, based on the target assessment result, the ISP parameter thereby improving subsequent image quality of the target image.
2 . The method of claim 1 , wherein the target assessment result comprises: the objective quality information of the target image that indicates a first assessed quality status of the target image in terms of an objective indicator; and the subjective quality information of the target image that indicates a second assessed quality status of the target image in terms of subjective aesthetics.
3 . The method of claim 2 , wherein the objective quality information comprises an objective score value corresponding to a specified dimension, wherein the subjective quality information comprises a subjective score value corresponding to the specified dimension, and wherein the specified dimension comprises at least one of a brightness dimension, a color dimension, a contrast ratio dimension, or a definition and noise dimension.
4 . The method of claim 3 , wherein the ISP parameter comprises at least one of brightness, a color, a contrast ratio, definition, or noise, and wherein adjusting the ISP parameter comprises:
determining, for each specified dimension and based on the objective score value and the subjective score value that correspond to the specified dimension, a target score value corresponding to the specified dimension; and
adjusting, based on the target score value and a preset parameter threshold, the ISP parameter corresponding to the specified dimension, wherein the preset parameter threshold is of the specified dimension in terms of a scene type indicated by a scene classification result.
5 . The method of claim 1 , wherein before invoking the pre-trained image quality assessment model, the method further comprises obtaining the pre-trained image quality assessment model through training based on at least one sample data group, and wherein each sample data group comprises the sample image and a pre-labeled correct assessment result.
6 . The method of claim 5 , wherein before obtaining the pre-trained image quality assessment model, the method further comprises:
obtaining a training sample set comprising the at least one sample data group;
inputting, for each of the at least one sample data group, the sample image to an original parameter model to obtain a training result, wherein the original parameter model is a multi-task neural network model;
comparing the training result with the pre-labeled correct assessment result to obtain a calculation loss that indicates an error between the training result and the pre-labeled correct assessment result; and
obtaining the pre-trained image quality assessment model through training based on the calculation loss corresponding to each of the at least one sample data group.
7 . The method of claim 6 , wherein after invoking the pre-trained image quality assessment model, the method further comprises:
performing automatic labeling on the target image to obtain label information comprising an objective label value corresponding to a specified dimension;
performing, based on the label information, screening in the training sample set to obtain a screened training sample set; and
training, based on the screened training sample set, the pre-trained image quality assessment model to obtain an updated image quality assessment model.
8 . An apparatus, comprising: a memory configured to store programming instructions; and a processor coupled to the memory and configured to execute the programming instructions to cause the apparatus to: process, based on an image signal processor (ISP) parameter, a captured original image to obtain a target image; invoke, based on the target image, a pre-trained image quality assessment model to obtain a target assessment result, wherein the pre-trained image quality assessment model comprises a scene classification layer, an objective scoring layer, wherein the pre-trained image quality assessment model is from training a neural network by using a sample image, and a subjective scoring layer, wherein the target assessment result indicates image quality of the target image, and wherein invoking the pre-trained image quality assessment model comprises: inputting the target image to the scene classification layer to obtain a scene classification result; inputting the target image and the scene classification result to the objective scoring layer to obtain objective quality information; and inputting the target image and the scene classification result to the subjective scoring layer to obtain subjective quality information; and adjust, based on the target assessment result, the ISP parameter thereby improving subsequent image quality of the target image.
9 . The apparatus of claim 8 , wherein the target assessment result comprises: the objective quality information of the target image that indicates a first assessed quality status of the target image in terms of an objective indicator; and the subjective quality information of the target image that indicates a second assessed quality status of the target image in terms of subjective aesthetics.
10 . The apparatus of claim 8 , wherein the objective quality information comprises an objective score value corresponding to a specified dimension, wherein the subjective quality information comprises a subjective score value corresponding to the specified dimension, and wherein the specified dimension comprises at least one of a brightness dimension, a color dimension, a contrast ratio dimension, or a definition and noise dimension.
11 . The apparatus of claim 10 , wherein the ISP parameter comprises at least one of brightness, a color, a contrast ratio, definition, or noise, and wherein the processor is further configured to execute the programming instructions to cause the apparatus to:
determine, for each specified dimension and based on the objective score value and the subjective score value that correspond to the specified dimension, a target score value corresponding to the specified dimension; and
adjust, based on the target score value and a preset parameter threshold, the ISP parameter corresponding to the specified dimension, wherein the preset parameter threshold is of the specified dimension in terms of a scene type indicated by a scene classification result.
12 . The apparatus of claim 8 , wherein the processor is further configured to execute the programming instructions to cause the apparatus to obtain the pre-trained image quality assessment model, wherein the pre-trained image quality assessment model is obtained through training based on at least one sample data group, and wherein each sample data group comprises the sample image and a pre-labeled correct assessment result.
13 . The apparatus of claim 12 , wherein processor is further configured to execute the programming instructions to cause the apparatus to:
obtain a training sample set comprising the at least one sample data group;
input, for each of the at least one sample data group, the sample image to an original parameter model to obtain a training result, wherein the original parameter model is a multi-task neural network model;
compare the training result with the pre-labeled correct assessment result to obtain a calculation loss that indicates an error between the training result and the pre-labeled correct assessment result; and
obtain the pre-trained image quality assessment model through training based on the calculation loss corresponding to each of the at least one sample data group.
14 . The apparatus of claim 13 , wherein the processor is further configured to execute the programming instructions to cause the apparatus to:
perform automatic labeling on the target image to obtain label information comprising an objective label value corresponding to a specified dimension;
perform, based on the label information, screening in the training sample set to obtain a screened training sample set; and
train, based on the screened training sample set, the pre-trained image quality assessment model to obtain an updated image quality assessment model.
15 . A computer program product comprising computer-executable instructions that are stored on a non-transitory computer-readable medium and that, when executed by a processor, cause an apparatus to: process, based on an image signal processor (ISP) parameter, a captured original image to obtain a target image; invoke, based on the target image, a pre-trained image quality assessment model to obtain a target assessment result, wherein the pre-trained image quality assessment model is from training a neural network by using a sample image, wherein the pre-trained image quality assessment model comprises a scene classification layer, an objective scoring layer, and a subjective scoring layer, wherein the target assessment result indicates image quality of the target image, and wherein invoking the pre-trained image quality assessment model comprises: inputting the target image to the scene classification layer to obtain a scene classification result; inputting the target image and the scene classification result to the objective scoring layer to obtain objective quality information; and inputting the target image and the scene classification result to the subjective scoring layer to obtain subjective quality information; and adjust, based on the target assessment result, the ISP parameter thereby improving subsequent image quality of the target image.
16 . The computer program product of claim 15 , wherein the target assessment result comprises: the objective quality information of the target image that indicates a first assessed quality status of the target image in terms of an objective indicator; and the subjective quality information of the target image that indicates a second assessed quality status of the target image in terms of subjective aesthetics.
17 . The computer program product of claim 15 , wherein the objective quality information comprises an objective score value corresponding to a specified dimension, wherein the subjective quality information comprises a subjective score value corresponding to the specified dimension, and wherein the specified dimension comprises at least one of a brightness dimension, a color dimension, a contrast ratio dimension, or a definition and noise dimension.
18 . The computer program product of claim 15 , wherein before invoking the pre-trained image quality assessment model, the computer-executable instructions further cause the apparatus to obtain the pre-trained image quality assessment model through training based on at least one sample data group, and wherein each sample data group comprises the sample image and a pre-labeled correct assessment result.
19 . The computer program product of claim 18 , wherein the computer-executable instructions when executed by the processor further cause the apparatus to: obtain a training sample set comprising the at least one sample data group; input, for each of the at least one sample data group, the sample image to an original parameter model to obtain a training result, wherein the original parameter model is a multi-task neural network model; compare the training result with the pre-labeled correct assessment result to obtain a calculation loss that indicates an error between the training result and the pre-labeled correct assessment result; and obtain the pre-trained image quality assessment model through training based on the calculation loss corresponding to each of the at least one sample data group.
20 . The computer program product of claim 19 , wherein the computer-executable instructions when executed by the processor further cause the apparatus to: perform automatic labeling on the target image to obtain label information comprising an objective label value corresponding to a specified dimension;
perform, based on the label information, screening in the training sample set to obtain a screened training sample set; and
train, based on the screened training sample set, the pre-trained image quality assessment model to obtain an updated image quality assessment model.