Systems and methods for acne counting, localization and visualization
Systems, methods and techniques provide for acne localization, counting and visualization. An image is processed using a trained model to identify objects. The model may be a deep learning (e.g. convolutional neural) network configured for object classification with a detection focus on small objects. The image may be a frontal or profile facial image, processed end to end. The model identifies and localizes different types of acne. Instances are counted and visualized such as by annotating the source image. An example annotation is an overlay identifying a type and location of each instance. Counts by acne type assist with scoring. A product and/or service may be recommended in response to the identification of the acne (e.g. the type, localization, counting and/or a score).
1. A method for analyzing skin images comprising: analyzing a source image using a convolutional neural network (CNN) model configured to process the source image pixel-by-pixel and output anchor boxes indicating respective locations of instances of acne;
visualizing the instances of acne on the source image for display by overlaying annotations on the source image showing the location and type of each acne instance based on output from the CNN model;
wherein the CNN model is trained using labelled facial images to:
detect at least one type of acne in images; and
focus detection on acne occurrences in images; and
wherein the CNN model comprises: a YOLO (You Only Look Once)-based model configured to comprise a single detection layer to form a single detection YOLO-based model and further configured to output a plurality of predictions comprising a three dimensional tensor encoding bounding box, objectness, and class predictions, and wherein the plurality of predictions are filtered to eliminate redundant detections of the same acne instance, the single detection layer YOLO-based model having been fine tuned to optimize acne localization accuracy.
2. The method of claim 1 , wherein the CNN model is a deep learning neural network model configured for object classification and localization.
3. The method of claim 1 , wherein the CNN model is configured to process the source image on a pixel level, operating end-to-end to directly detect acne location.
4. The method of claim 1 , wherein the CNN model generates respective anchor boxes providing location information for each of the instances of acne detected.
5. The method of claim 4 , wherein an anchor box aspect ratio, for use to define one of the anchor boxes, is calculated using k-means clustering from acne instances identified in a dataset of images.
6. The method of claim 1 , wherein visualizing the instances of the acne indicates a respective location for each of the instances on the source image and indicates a respective type of acne for each of the instances on the source image.
7. The method of claim 1 , wherein the at least one type of acne includes one or more of retentional acne, inflammatory acne and pigmentary acne.
8. The method of claim 1 comprising determining a count of the instances of acne.
9. The method of claim 1 comprising obtaining a recommendation for one or more of a product and a service specific to treat the instances of the acne.
10. The method of claim 9 comprising communicating with an e-commerce system for making a purchase of the product or service.
11. The method of claim 9 , wherein the recommendation is generated in response to factors selected from the group: acne type, count by type, score by type, location of the acne, location of purchaser, delivery location, regulatory requirement, counter indication, gender, co-recommendation, and likelihood of user to follow use guidelines.
12. A computing system for analyzing skin images comprising circuitry configured to provide:
an interface to receive a source image and return an annotated source image which visualizes instances of acne determined by a convolutional neural network (CNN) model configured to process the source image pixel-by-pixel and output anchor boxes indicating respective locations of instances of acne;
wherein the CNN model is configured to:
focus detection on small objects in images; and
detect at least one type of acne in images; and
wherein the CNN model comprises:
a single detection layer YOLO (You Only Look Once)-based model configured to output a plurality of predictions comprising a three dimensional tensor encoding bounding box, objectness, and class predictions, and wherein the plurality of predictions are filtered to eliminate redundant detections of the same acne instance, the single detection layer YOLO-based model being fine tuned to optimize acne localization accuracy.
13. The computing system of claim 12 configured to provide:
a recommendation component configured to recommend a product and/or service to specifically treat at least some of the instances of acne; and,
an e-commerce transaction component to facilitate a purchase of the product and/or service.
14. The computing system of claim 12 , wherein the single detection layer YOLO-based model is fine tuned by one or more of:
performing data augmentation on the labelled facial images comprising:
converting the labelled facial images to HSV color space;
adding random color jittering on saturation and value channels of the HSV color space images;
applying random affine transformation on the HSV color space images;
utilizing a multi-step learning rate scheduler during model training; or
employing an evolution algorithm to iteratively adjust parameters of the data augmentation, wherein the evolution algorithm comprises:
training the model through multiple rounds; and
selecting parameters that optimize acne detection.
15. The method of claim 1 , wherein the single detection layer YOLO-based model is fine tuned by one or more of:
performing data augmentation on the labelled facial images comprising:
converting the labelled facial images to HSV color space;
adding random color jittering on saturation and value channels of the HSV color space images;
applying random affine transformation on the HSV color space images;
utilizing a multi-step learning rate scheduler during model training; or
employing an evolution algorithm to iteratively adjust parameters of the data augmentation, wherein the evolution algorithm comprises:
training the model through multiple rounds; and
selecting parameters that optimize acne detection.