System and method for guiding a user to take a selfie
Systems and methods for improving the image quality of a selfie. The system and method include analyzing a selfie with image processing modules and selfie quality index modules. The image processing modules process the image using face and landmark detection techniques and face normalization techniques. The selfie quality index modules analyze the selfie for important image quality metrics and provide feature vectors that are used to generate a selfie quality index score. The selfie quality index score can be used to determine whether the selfie meet minimum threshold quality requirement for further analysis by the system for a skin condition.
1. A system for improving the quality of a selfie, comprising:
a. an image capture device;
b. a computer coupled to the image capture device, the computer comprising a non-transitory memory component with logic stored thereon, wherein the logic comprises
i) an image processing module that causes the computer to locate a face in a selfie captured by the image capture device and provided to the computer, locate at least one landmark facial feature present on the face, and normalize the selfie, and
ii) a selfie quality index (SQI) module that causes the computer to analyze the normalized selfie for a quality metric selected from occlusion, blur, distance-to-camera, facial expression, illumination, and combinations of these, wherein the system uses the analyzed image quality metric to generate an SQI score,
iii) wherein the SQI module comprises a Blur Detection Module in the form of a CNN trained by a blur feature vector formed from at least one of a blur coefficient and a feature extracted from the last layer of an Alexnet CNN, wherein the blur coefficient is generated by taking the differences between Laplacian measures on the normalized image after several consecutive rounds of blurring; and
c. a display coupled to at least one of the image capture device and the computing device.
2. The system of claim 1 , wherein the image processing module comprises a Face Detection and Counting Module that isolates at least a portion of the face from background features present in the selfie using a bounding box.
3. The method of claim 2 , wherein the SQI module comprises a Distance to Camera Module that determines the distance-to-camera image quality metric by calculating a ratio of pixels in the bounding box to image size.
4. The method of claim 3 , wherein the logic causes the computer to communicate to a user that the image capture device is too close when the ratio of pixels in the bounding box to image size is greater than 0.8.
5. The method of claim 3 , wherein the logic causes the computer to communicate to a user that the image capture device is too far away when the ratio of pixels in the bounding box to image size is less than 0.4.
6. The system of claim 1 , wherein the image processing module comprises a Landmark Detection Module in the form of a convolutional neural network (CNN) to determine if the at least one landmark is present.
7. The method of claim 6 , wherein the SQI module comprises a Distance to Camera Module that determines the distance-to-camera image quality metric by comparing a pixel based distance between at least two landmark points in the selfie to a known average distance between said landmark points for a population.
8. The system of claim 1 , wherein the image processing module comprises a Face Registration Module that causes the system to normalize the face based on positions of at least 4 facial landmark points.
9. The system of claim 1 , wherein the SQI module comprises an Occlusion Module that analyzes the normalized image for the occlusion image quality metric, and wherein the Occlusion Module generates an occlusion feature vector formed by fusing at least one local binary pattern feature with at least one CNN-based Alexnet feature.
10. The method of claim 1 , wherein the SQI module comprises an Expression Module that analyzes the normalized selfie for the expression image quality metric, and wherein the Expression Module applies a local binary pattern, histogram of gradients, deep learning technique, or combination of these to determine an expression feature vector.
11. The method of claim 1 , wherein the SQI module comprises an Illumination Module that analyzes the normalized selfie for the illumination image quality metric.
12. The method of claim 11 , wherein Illumination Module includes at least one sub-module for determining a direction of illumination, an intensity of illumination, contrast, and location of illumination.