Face region detection and local reshaping enhancement
Methods and corresponding systems to process face regions are disclosed. The described methods include providing face bounding boxes and confidence levels for the faces, generating a histogram of the pixels and the faces, generating a probability of face, and generating a face probability map. A face contrast adjustment and a face saturation adjustment can be applied to the face probability map.
1 . A method of performing local reshaping on an input image including one or more faces, the method comprising: generating a histogram of all pixels in the input image; based on a combination of face bounding boxes for the one or more faces with a basic face shape model, generating a basic face shape map comprising a pixel mapping of basic face shapes for the one or more faces; based on the input image and the basic face shape map, generating histograms of the one or more faces; based on the histograms of the one more faces, generating for each bin of the histogram of all pixels a probability of face comprising a probability of a pixel being in a face, based on the probability of face, generating a face probability map comprising a pixel mapping of the input image to the probabilities of each pixel individually being part of a face, and generating a reshaped image from the input image based on the face probability map and one or more selected reshaping functions, wherein after generating the probability of face and before generating the face probability map, local smoothing the probability of face to generate a smoothened probability of face, and applying a soft morphological operation to the smoothened probability of face to generate the face probability map.
2 . The method of claim 1 , wherein the basic face shape model comprises an inscribed ellipse of a bounding box.
3 . The method of any of claim 1 , wherein the generating of the probability of face comprises:
filtering the histogram of all pixels to generate a filtered histogram of all pixels, and
filtering the histograms of the one or more faces to generated filtered histograms of the one or more faces.
4 . The method of claim 3 , further wherein the generating of the probability of face further comprises:
scaling and thresholding a combination of the filtered histogram of all pixels and filtered histograms of the one or more faces to generate an initial probability of face.
5 . The method of claim 4 , wherein the initial probability of face comprises an initial probability of face in YUV channel.
6 . The method of claim 3 , wherein:
the filtering the histogram of all pixels is performed using a gaussian filter, and
the filtering the histograms of the one or more faces is performed using a gaussian filter.
7 . The method of claim 4 , wherein the combination of the filtered histogram of all pixels and filtered histograms of the one or more faces comprises a ratio of the filtered histograms of the one or more faces with the filtered histogram of all pixels.
8 . The method of claim 4 , wherein the generating of the probability of face further comprises:
subtracting the generated histograms of the one or more faces from the generated histogram of all pixels to generate a histogram of non-face.
9 . The method of claim 8 , wherein generating of the probability of face further comprises:
based on the initial probability of face and the histogram of non-face, generating an updated probability of non-face, and
based on the initial probability of face and the histograms of the one or more faces, generating an updated probability of face.
10 . The method of claim 9 , generating of the probability of face further comprises:
combining the updated probability from non-face and the updated probability from face to generate an updated probability, and
filtering the updated probability to generate the probability of face.
11 . The method of claim 10 , wherein the filtering is performed using a gaussian filter.
12 . The method of claim 10 , further comprising:
after generating the probability of face and before generating the face probability map, local smoothing the probability of face to generate a smoothened probability of face, and
applying a soft morphological operation to the smoothened probability of face to generate the face probability map.
13 . The method of claim 10 , wherein the combining the updated probability from non-face and the updated probability from face comprises:
generating a weighted sum of updated probability from non-face and the updated probability from face.
14 . The method of claim 1 further comprising:
applying local reshaping by:
applying face saturation adjustment and face contrast adjustment to the face probability map to generate an adjusted face probability map; and
generating a reshaped image based on the adjusted face probability map and one or more selected reshaping function.
15 . The method of claim 14 , wherein the applying the face contrast adjustment is performed by adjusting a contrast of the one or more faces based on a face contrast reduction ratio.
16 . The method of claim 14 , wherein the applying the face saturation adjustment is performed by adjusting a saturation of the one or more faces based a face desaturation offset and a face desaturation threshold.
17 . The method of claim 1 , further comprising:
trimming the histograms of the one or more faces to reduce a memory space required to store the histograms of the one or more faces.
18 . A video decoder comprising hardware, software, or both configured to carry out the method of claim 1 .
19 . A non-transitory computer-readable storage medium having stored thereon computer-executable instruction for executing a method with one or more processors in accordance with claim 1 .