IP Library Granted Patent US 10,255,484
Granted Patent B2
US 10,255,484 · App. 15/414,095 · Granted Apr 9, 2019

Method and system for assessing facial skin health from a mobile selfie image

Inventors: Martin S. Maltz (Rochester, NY); Matthew Adam Shreve (Webster, NY); Luisa Fernanda Polania Cabrera (Webster, NY); Stephen C. Morgana (Rochester, NY); Raja Bala (Pittsford, NY); Paul Jonathan Matts (Surrey, GB); Ankur Purwar (Sunglade, SG)
Assignee: The Procter & Gamble Company
G06K9/00281A61B5/0022A61B5/0077A61B5/443A61B5/486A61B5/742G06Q30/0631G06T7/40G06T7/90G09B19/00G06K2009/4666G06T2207/10024G06T2207/30201H04N1/00106
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,255,484
App. No.
15/414,095
Granted
Apr 9, 2019
Kind
B2
Abstract

A method, non-transitory computer readable medium and apparatus for transmitting an assessment of facial skin health of a customer are disclosed. For example, the method includes receiving an image of a customer from a mobile endpoint device of the customer, analyzing one or more parameters of a facial skin of the customer in the image of the customer, determining the assessment of facial skin health of the customer based on the one or more parameters of the facial skin that are analyzed and transmitting the assessment of facial skin health to the mobile endpoint device of the customer to cause the mobile endpoint device to automatically display the assessment of facial skin health.

Claims (36)

1. A method for transmitting an assessment of facial skin health of a customer, comprising:

receiving, by a processor, an image of the customer from a mobile endpoint device of the customer;

analyzing, by the processor, one or more parameters of a facial skin of the customer in each pixel of the image of the customer;

determining, by the processor, the assessment of facial skin health of the each pixel of the image of the customer based on the one or more parameters of the facial skin that are analyzed;

associating, by the processor, a corresponding color to the each pixel based on the assessment of facial skin health that is determined for the each pixel;

generating, by the processor, a visual map based on the corresponding color that is associated with the each pixel based on the assessment of facial skin health that is determined for the each pixel, wherein the visual map comprises a plurality of facial regions that include concentric areas having a different color based on the corresponding color that is associated with the each pixel; and

transmitting, by the processor, the assessment of facial skin health that includes the visual map to the mobile endpoint device of the customer to cause the mobile endpoint device to automatically display the assessment of facial skin health.

2. The method of claim 1 , wherein the analyzing the one or more parameters is performed for each one of a plurality of different facial regions.

3. The method of claim 1 , wherein the one or more parameters comprise a melanin content of the facial skin, a hemoglobin content of the facial skin, one or more wrinkles of the facial skin, one or more pores of the facial skin and a texture of the facial skin.

4. The method of claim 3 , wherein the melanin content of the facial skin and the hemoglobin content of the facial skin are analyzed based upon a modified independent component analysis performed on the image to obtain a melanin component and a hemoglobin component from the image.

5. The method of claim 3 , wherein the one or more wrinkles are identified by applying a Hessian based Frangi Filtering operation on the image.

6. The method of claim 3 , wherein the texture of the facial skin is obtained based on a local binary patterns analysis and a random forest regression analysis applied to the image.

7. The method of claim 1 , wherein the assessment of facial skin health comprises a binary outcome based upon each one of the one or more parameters being above a respective threshold value.

8. The method of claim 1 , wherein the assessment of facial skin health comprises a numerical score based upon a weighted score of each one of the one or more parameters.

9. A non-transitory computer-readable medium storing a plurality of instructions, which when executed by a processor, cause the processor to perform operations for transmitting an assessment of facial skin health of a customer, the operations comprising:

receiving an image of the customer from a mobile endpoint device of the customer;

analyzing one or more parameters of a facial skin of the customer in each pixel of the image of the customer;

determining the assessment of facial skin health of the each pixel of the image of the customer based on the one or more parameters of the facial skin that are analyzed;

associating a corresponding color to the each pixel based on the assessment of facial skin health that is determined for the each pixel;

generating a visual map based on the corresponding color that is associated with the each pixel based on the assessment of facial skin health that is determined for the each pixel, wherein the visual map comprises a plurality of facial regions that include concentric areas having a different color based on the corresponding color that is associated with the each pixel; and

transmitting the assessment of facial skin health that includes the visual map to the mobile endpoint device of the customer to cause the mobile endpoint device to automatically display the assessment of facial skin health.

10. The non-transitory computer-readable medium of claim 9 , wherein the analyzing the one or more parameters is performed for each one of a plurality of different facial regions.

11. The non-transitory computer-readable medium of claim 9 , wherein the one or more parameters comprise a melanin content of the facial skin, a hemoglobin content of the facial skin, one or more wrinkles of the facial skin, one or more pores of the facial skin and a texture of the facial skin.

12. The non-transitory computer-readable medium of claim 11 , wherein the melanin content of the facial skin and the hemoglobin content of the facial skin are analyzed based upon a modified independent component analysis performed on the image to obtain a melanin component and a hemoglobin component from the image.

13. The non-transitory computer-readable medium of claim 11 , wherein the one or more wrinkles are identified by applying a Hessian based Frangi Filtering operation on the image.

14. The non-transitory computer-readable medium of claim 11 , wherein the texture of the facial skin is obtained based on a local binary patterns analysis and a random forest regression analysis applied to the image.

15. The non-transitory computer-readable medium of claim 9 , wherein the assessment of facial skin health comprises a binary outcome based upon each one of the one or more parameters being above a respective threshold value.

16. The non-transitory computer-readable medium of claim 9 , wherein the assessment of facial skin health comprises a numerical score based upon a weighted score of each one of the one or more parameters.

17. A method for transmitting an assessment of facial skin health of a customer, comprising:

receiving, by a processor, an image of the customer from a mobile endpoint device of the customer;

analyzing, by the processor, a melanin content, a hemoglobin content, one or more wrinkles, one or more pores and a texture for each one of a plurality of different facial regions of a facial skin of the customer in each pixel of the image of the customer;

calculating, by the processor, a respective score for the melanin content, the hemoglobin content, the one or more wrinkles, the one or more pores and the texture based on the analyzing;

determining, by the processor, the assessment of facial skin health of the each pixel of the image of the customer based on the respective score that is calculated for the melanin content, the hemoglobin content, the one or more wrinkles, the one or more pores and the texture of the facial skin;

associating, by the processor, a corresponding color to the each pixel based on the assessment of facial skin health that is determined for the each pixel;

generating, by the processor, a visual map based on the corresponding color that is associated with the each pixel based on the assessment of facial skin health that is determined for the each pixel, wherein the visual map comprises a plurality of facial regions that include concentric areas having a different color based on the corresponding color that is associated with the each pixel; and

transmitting, by the processor, the assessment of facial skin health that includes the visual map to the mobile endpoint device of the customer to cause the mobile endpoint device to automatically display the assessment of facial skin health.

Assignments (11)
SECOND LIEN NOTES PATENT SECURITY AGREEMENT Recorded Jul 2, 2025
From: XEROX CORPORATION
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 071785/0550 →
FIRST LIEN NOTES PATENT SECURITY AGREEMENT Recorded Apr 11, 2025
From: XEROX CORPORATION
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 070824/0001 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS RECORDED AT RF 064760/0389 Recorded Feb 13, 2024
From: CITIBANK, N.A., AS COLLATERAL AGENT
To: XEROX CORPORATION
Reel/Frame 068261/0001 →
SECURITY INTEREST Recorded Feb 13, 2024
From: XEROX CORPORATION
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 066741/0001 →
SECURITY INTEREST Recorded Nov 20, 2023
From: XEROX CORPORATION
To: JEFFERIES FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 065628/0019 →
CORRECTIVE ASSIGNMENT TO CORRECT THE REMOVAL OF US PATENTS 9356603, 10026651, 10626048 AND INCLUSION OF US PATENT 7167871 PREVIOUSLY RECORDED ON REEL 064038 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jun 28, 2023
From: PALO ALTO RESEARCH CENTER INCORPORATED
To: XEROX CORPORATION
Reel/Frame 064161/0001 →
SECURITY INTEREST Recorded Jun 22, 2023
From: XEROX CORPORATION
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 064760/0389 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2023
From: PALO ALTO RESEARCH CENTER INCORPORATED
To: XEROX CORPORATION
Reel/Frame 064038/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 8, 2018
From: XEROX CORPORATION
To: PROCTER & GAMBLE COMPANY
Reel/Frame 045473/0802 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 19, 2017
From: MATTS, PAUL JONATHAN; PURWAR, ANKUR
To: XEROX CORPORATION
Reel/Frame 043622/0402 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 1, 2017
From: MALTZ, MARTIN S.; SHREVE, MATTHEW ADAM; POLANIA CABRERA, LUISA FERNANDA; MORGANA, STEPHEN C.; BALA, RAJA
To: XEROX CORPORATION
Reel/Frame 041145/0356 →
Continuity (2)
Provisional Application 62311192 · Mar 21, 2016
Related Publication 20170270350A1 · Sep 21, 2017