Techniques for identifying skin color in images having uncontrolled lighting conditions
In some embodiments of the present disclosure, one or more machine learning models are trained to accurately estimate skin color in one or more images regardless of the lighting conditions. In some embodiments, the models can then be used to estimate a skin color in a new image, and that estimated skin color can be used for a variety of purposes. For example, the skin color may be used to generate a recommendation for a foundation shade that accurately matches the skin color, or a recommendation for another cosmetic product that is complimentary with the estimated skin color. Thus, the need for an in-person test of the product is eliminated.
1. A method of training a machine learning model to estimate a skin color of a face in an image, the method comprising:
receiving, by a computing device, at least one training image that includes a face of a training subject and a color reference chart;
receiving, by the computing device, tagging information for the at least one training image, wherein receiving the tagging information for the at least one training image includes:
detecting, by the computing device, the color reference chart from the at least one training image that includes the face of the training subject; and
determining, by the computing device, an illuminant color based on the color reference chart;
adding, by the computing device, the at least one training image and the tagging information to a training data store; and
training, by the computing device, the machine learning model to determine skin colors of faces using information stored in the training data store.
2. The method of claim 1 , further comprising normalizing, by the computing device, the at least one training image to create at least one normalized training image.
3. The method of claim 2 , wherein normalizing the at least one training image to create at least one normalized training image includes:
detecting, by the computing device, a face in the at least one training image;
centering, by the computing device, the face in the at least one training image; and
zooming, by the computing device, the at least one training image such that the face is a predetermined size.
4. The method of claim 1 , wherein receiving at least one training image includes:
receiving, by the computing device, a video; and
extracting, by the computing device, at least one training image from the video.
5. The method of claim 1 , further comprising adjusting, by the computing device, a color of the at least one training image based on the determined illuminant color.
6. The method of claim 1 , wherein training the machine learning model to determine skin colors of faces using the training data set includes:
training, by the computing device, a first machine learning model that processes a training image as input to produce an indication of a lighting condition as an output; and
training, by the computing device, a second machine learning model that processes the training image and the indication of the lighting condition as input to produce an indication of skin color as an output.
7. A non-transitory computer-readable medium having computer-executable instructions stored thereon that, in response to execution by one or more processors of a computing device, cause the computing device to perform actions for training a machine learning model to estimate a skin color of a face in an image, the actions comprising:
receiving, by the computing device, at least one training image that includes a face of a training subject and a color reference chart;
receiving, by the computing device, tagging information for the at least one training image, wherein receiving the tagging information for the at least one training image includes:
detecting, by the computing device, the color reference chart from the at least one training image that includes the face of the training subject; and
determining, by the computing device, an illuminant color based on the color reference chart;
adding, by the computing device, the at least one training image and the tagging information to a training data store; and
training, by the computing device, the machine learning model to determine skin colors of faces using information stored in the training data store.
8. The computer-readable medium of claim 7 , wherein the actions further comprise normalizing, by the computing device, the at least one training image to create at least one normalized training image.
9. The computer-readable medium of claim 8 , wherein normalizing the at least one training image to create at least one normalized training image includes:
detecting, by the computing device, a face in the at least one training image;
centering, by the computing device, the face in the at least one training image; and
zooming, by the computing device, the at least one training image such that the face is a predetermined size.
10. The computer-readable medium of claim 7 , wherein receiving at least one training image includes:
receiving, by the computing device, a video; and
extracting, by the computing device, at least one training image from the video.
11. The computer-readable medium of claim 7 , wherein the actions further comprise adjusting, by the computing device, a color of the at least one training image based on the determined illuminant color.
12. The computer-readable medium of claim 7 , wherein training the machine learning model to determine skin colors of faces using the training data set includes:
training, by the computing device, a first machine learning model that processes a training image as input to produce an indication of a lighting condition as an output; and
training, by the computing device, a second machine learning model that processes the training image and the indication of the lighting condition as input to produce an indication of skin color as an output.
13. A system for training a machine learning model to estimate a skin color of a face in an image, the system comprising:
circuitry for receiving at least one training image that includes a face of a training subject and a color reference chart;
circuitry for receiving tagging information for the at least one training image, wherein receiving the tagging information for the at least one training image includes:
detecting the color reference chart from the at least one training image that includes the face of the training subject; and
determining an illuminant color based on the color reference chart;
circuitry for adding the at least one training image and the tagging information to a training data store; and
circuitry for training the machine learning model to determine skin colors of faces using information stored in the training data store.
14. The system of claim 13 , further comprising circuitry for normalizing the at least one training image to create at least one normalized training image.
15. The system of claim 14 , wherein normalizing the at least one training image to create at least one normalized training image includes:
detecting a face in the at least one training image;
centering the face in the at least one training image; and
zooming the at least one training image such that the face is a predetermined size.
16. The system of claim 13 , wherein receiving at least one training image includes:
receiving a video; and
extracting at least one training image from the video.
17. The system of claim 13 , further comprising circuitry for adjusting a color of the at least one training image based on the determined illuminant color.
18. The system of claim 13 , wherein training the machine learning model to determine skin colors of faces using the training data set includes:
training a first machine learning model that processes a training image as input to produce an indication of a lighting condition as an output; and
training a second machine learning model that processes the training image and the indication of the lighting condition as input to produce an indication of skin color as an output.