IP Library › Granted Patent US 12,551,162
Granted Patent B2
US 12,551,162 · App. 17/931,108 · Granted Feb 17, 2026

Skin tone scanning and cosmetics in three-dimensional skin tone color space

Inventors: Prasad Tendulkar (Foster City, CA); Nelly Mensah (San Francisco, CA); Anna Teresa Nicholas (Hazelton, PA); Jillian Bertone (Lafayette, CA); Lisa Zhu (Belmont, CA); Jennifer Stokes (Seattle, WA); Witono Liujanto (San Ramon, CA); Aaron Oelckers (Pleasant Hill, CA); Marc McCotter (Hana, HI); Harsha Huddar (Livermore, CA); Fiona Kavanagh (San Francisco, CA); Amit Adiraju Narasimha (Santa Clara, CA); Naroju Janardhan (San Ramon, CA)
Assignee: Sephora USA, Inc.
A61B5/441A45D44/005A61B5/0064A61B5/442G06Q30/0621G06Q30/0631G06Q30/0633G06T7/0012G06T7/90G06V10/26G06V10/56G06V10/764G06V10/774G06V10/776G06V10/82G06V40/10A45D2044/007G06Q30/01G06Q30/0641G06T2200/24G06T2207/20081G06T2207/30088G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,551,162
App. No.
17/931,108
Granted
Feb 17, 2026
Kind
B2
Abstract

Scanning a person's skin allows one to more easily find products that match their skin tone. Cosmetic products are organized or classified using a three-dimensional skin tone color space. A skin scan determines a person's skin tone in the same skin tone color space. This allows customers to determine which products are appropriate for their skin tone that they can purchase form a retailer. A scanning device is used to scan one or more locations of a person's skin to obtain a skin tone identifier. For example, two different locations of a person's face can be scanned, such as the forehead and cheek. There may be an additional third scan. The scan determines a skin tone result in the skin tone color space.

Claims (63)

1 . A system comprising:

a plurality of cosmetic products placed on one or more shelves in a retail store, each cosmetic product having a printed tag associated with the each cosmetic product that is coupled to the shelf upon which the each cosmetic product is placed, wherein each printed tag comprises a printed skin tone identifier in a three-dimensional skin tone color space;

a handheld scanning device that comprises a mobile communication device and a sensor assembly, wherein the sensor assembly comprises a client side, a device side opposite to the client side, and one or more lenses on the client side, the mobile communication device comprises a scanner display on a front side, and a scanning sensor on a back side, opposite to the front side, and the device side of the sensor assembly is attached to the back side of the mobile communication device so that while the client side of the sensor assembly is directed toward a customer's skin of a customer, the scanning sensor is operatively coupled with the one or more lenses to perform scans of the customer's skin while the scanner display remains visible on the front side;

a battery, enclosed within the handheld scanning device, that powers the handheld scanning device; and

the handheld scanning device is configured to determine a final skin tone result, determining the final skin tone result comprising:

to determine the final skin tone result, while the client side of the sensor assembly is directed toward the customer's skin at a first location, performing a first scan to generate a first intermediate result reflective of the first location, wherein the final skin tone result for the customer is unavailable for display on the scanner display after the first scan and before a second scan has been performed,

while the client side of the sensor assembly is directed toward the customer's skin at a second location, performing the second scan to generate a second intermediate result reflective of the second location,

using the first and second intermediate results, calculating the final skin tone result that is reflective of the first and second locations, and

displaying the final skin tone result on the scanner display, wherein the final skin tone result references the three-dimensional skin tone color space.

2 . The system of claim 1 wherein each skin tone in the skin tone color space comprises a skin tone identifier comprising three values or parameters to indicate its position in the skin tone color space, and the client side of the sensor assembly attached to the back side of the mobile computing device is placed directly against the customer's skin for at least one of the first scan or the second scan.

3 . The system of claim 1 wherein the handheld scanning device determines the final skin tone result based on three or more scans.

4 . The system of claim 1 wherein the mobile computing device in the handheld scanning device comprises a smartphone, the scanning sensor comprises a camera lens of the smartphone, and the scanning sensor is operatively coupled to at least one lens of the one or more lenses of the sensor assembly to perform the scans of the customer's skin when the sensor assembly is attached to the mobile computing device.

5 . The system of claim 1 wherein images of one or more of cosmetic products that are in the retail store that are associated with the final skin tone result of the customer are shown on the scanner display.

6 . The system of claim 1 further comprising:

a server, accessible by the handheld scanning device, comprising a customer database specifically configured to store a plurality of customer skin tone results and a plurality of customer skin guide results associated with a plurality of customer identifiers.

7 . The system of claim 6 wherein the plurality of customer identifiers comprise an e-mail address.

8 . The system of claim 6 wherein the customer database is further specifically configured to

receive from the handheld scanning device a first skin tone result and a first customer identifier, and

update the customer database to store the first skin tone result associated with the first customer identifier.

9 . The system of claim 8 wherein for the first customer identifier, the server sends to an e-mail address corresponding to the first customer identifier the first skin tone result.

10 . The system of claim 8 wherein for the first customer identifier, the server sends to an e-mail address corresponding to the first customer identifier a link to a screen that shows one or more cosmetic products available in the retail store that correspond to the first skin tone result.

11 . The system of claim 1 , further comprising:

the handheld scanning device is configured to determine a final skin condition result, determining the final skin condition result comprising:

to determine the final skin condition result, while the client side of the sensor assembly is directed toward the customer's skin at a third location, performing a third scan to generate a third intermediate result reflective of the third location, wherein the final skin condition result for the customer is unavailable for display on the scanner display after the third scan and before a fourth scan has been performed,

after the client side of the sensor assembly is directed toward the customer's skin at a fourth location, performing the fourth scan to generate a fourth intermediate result reflective of the fourth location,

using the third and fourth intermediate results, calculating the final skin condition result that is reflective of the third and fourth locations, and

displaying the final skin condition result on the scanner display.

12 . The system of claim 1 wherein the scanning sensor is placed against the customer's skin at the first location.

13 . The system of claim 6 wherein the retail store comprises the plurality of cosmetic products, the one or more shelves, and the handheld scanning device, and the server is located outside of the retail store and is coupled to the handheld scanning device via a network.

14 . A system comprising:

a plurality of cosmetic products placed on one or more shelves in a retail store, each cosmetic product having a shelf tag associated with the each cosmetic product that is coupled to a shelf upon which the each cosmetic product is placed, wherein each shelf tag comprises a printed skin tone identifier in a three-dimensional skin tone color space;

a handheld scanning device that comprises a mobile communication device and a sensor assembly, wherein the handheld scanning device comprises a mobile communication device and a sensor assembly, the sensor assembly comprises a client side, a device side opposite to the client side, and one or more lenses on the client side, the mobile communication device comprises a scanner display on a front side and a scanning sensor on a back side, opposite to the front side, and the device side of the sensor assembly is attached to the back side of the mobile communication device so that while client side of the sensor assembly is directed toward a customer's skin of a customer, the scanning sensor and the one or more lenses are operatively coupled to perform scans of the customer's skin while the scanner display remains visible on the front side;

a battery, enclosed within the handheld scanning device, that powers the handheld scanning device;

the handheld scanning device is configured to determine a final skin tone result, determining the final skin tone result comprising:

to determine the final skin tone result, while the client side of the sensor assembly is directed toward the a customer's skin at a first location, performing a first scan to generate a first intermediate result reflective of the first location, wherein the final skin tone result for the customer is unavailable for display on the scanner display after the first scan and before a second scan has been performed,

while the client side of the sensor assembly is directed toward the customer's skin at a second location, performing the second scan to generate a second intermediate result reflective of the second location,

using the first and second intermediate results, calculating the final skin tone result that is reflective of the first and second locations, and

displaying the final skin tone result on the scanner display, wherein the final skin tone result references the three-dimensional skin tone color space, and each skin tone in the skin tone color space comprises three values or parameters to indicate its position in the skin tone color space; and

the handheld scanning device is configured to determine a final skin condition result, determining the final skin condition comprising:

to determine the final skin condition result, while the client side of the sensor assembly is directed toward the customer's skin at a third location, performing a third scan to generate a third intermediate result reflective of the third location, wherein the final skin condition result for the customer is unavailable for display on the scanner display after the third scan and before a fourth scan has been performed,

while the client side of the sensor assembly is directed toward the customer's skin at a fourth location, performing the fourth scan to generate a fourth intermediate result reflective of the fourth location,

using the third and fourth intermediate results, calculating the final skin condition result that is reflective of the third and fourth locations, and

displaying the final skin condition result on the scanner display.

15 . The system of claim 14 , wherein the handheld scanning device determines the final skin condition result based on three or more scans, and the client side of the sensor assembly attached to the back side of the mobile computing device is placed directly against the customer's skin for at least one of the first, the second, the third, or the fourth scan.

16 . The system of claim 14 , wherein the final skin condition result comprises at least one of an indication of a skin type, a degree of oiliness or oil balance of the customer's skin, a pore size of the customer's skin, a degree of lines or wrinkles on the customer's skin, or a degree of hydration of the customer's skin, or any combination of these.

17 . The system of claim 12 , wherein images of one or more of products that are in the retail store and are associated with the final skin condition result of the customer are shown on the scanner display, and the scanning sensor is operatively coupled to at least one lens of the one or more lenses of the sensor assembly to perform the scans of the customer's skin when the sensor assembly is attached to the mobile computing device.

18 . A system comprising:

a plurality of cosmetic products placed on one or more shelves in a retail store, each cosmetic product having a shelf tag associated with the each cosmetic product that is coupled to an edge of a shelf upon which the each cosmetic product is placed, wherein each shelf tag comprises a printed skin tone identifier in a three-dimensional skin tone color space;

a handheld scanning device comprising a smartphone and a sensor assembly, wherein the sensor assembly comprises a client side, a device side opposite to the client side, and one or more lenses on the client side, the smartphone comprises a scanner display on a front side and a scanning sensor on a back side, opposite to the front side, so that while the client side of the sensor assembly is directed toward a customer's skin of a customer, the scanning sensor and the one or more lenses are operatively coupled to perform scans of the customer's skin while the scanner display remains visible on the front side;

a battery, enclosed within the handheld scanning device, that powers the handheld scanning device;

the sensor assembly comprises lens and the device side of the sensor assembly is attached to the back side the smartphone such that at least one of the one or more lens of the sensor assembly is aligned with the scanning sensor of the smartphone;

the handheld scanning device is configured to determine a final skin tone result, determining the final skin tone result comprising:

to determine the final skin tone result, while the client side of the sensor assembly is directed toward a customer's skin of a customer at a first location, performing a first scan to generate a first intermediate result reflective of the first location, wherein the final skin tone result for the customer is unavailable for display on the scanner display after the first scan and before a second scan has been performed,

while the client side of the sensor assembly is directed toward the customer's skin at a second location, performing the second scan to generate a second intermediate result reflective of the second location,

using the first and second intermediate results, calculating the final skin tone result that is reflective of the first and second locations, and

displaying the final skin tone result on the scanner display, wherein the final skin tone result references the three-dimensional skin tone color space, and a skin tone in the skin tone color space comprises three values or parameters to indicate its position in the skin tone color space; and

the handheld scanning device is configured to determine a final skin condition result, determining the final skin condition result comprising:

to determine the final skin condition result, while the client side of the sensor assembly is directed toward the customer's skin at a third location, performing a third scan to generate a third intermediate result reflective of the third location, wherein the final skin condition result for the customer is unavailable for display on the scanner display after the third scan and before a fourth scan has been performed,

while the client side of the sensor assembly is directed toward the customer's skin at a fourth location, performing the fourth scan to generate a fourth intermediate result reflective of the fourth location,

using the third and fourth intermediate results, calculating the final skin condition result that is reflective of third and fourth locations, and

displaying the final skin condition result on the scanner display.

19 . The system of claim 18 wherein the sensor assembly comprises two or more light-emitting diodes, a first light emitting diode that emits light of a first wavelength and a second light emitting diode that emits light of a second wavelength, which is different from the first wavelength.

20 . The system of claim 19 wherein the sensor assembly is wirelessly coupled to the smartphone via at least one of a Bluetooth connection or a near-field communication (NFC) connection.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 3, 2025
From: TENDULKAR, PRASAD; MENSAH, NELLY; BERTONE, JILLIAN; ZHU, LISA; STOKES, JENNIFER; LIUJANTO, WITONO; OELCKERS, AARON; MCCOTTER, MARC; HUDDAR, HARSHA; KAVANAGH, FIONA; NARASIMHA, AMIT ADIRAJU; JANARDHAN, NAROJU; NICHOLAS, ANNA TERESA
To: SEPHORA USA, INC.
Reel/Frame 072468/0703 →
Continuity (2)
Provisional Application 63261060 · Sep 9, 2021
Related Publication 20230115395A1 · Apr 13, 2023
References Cited (90)
US 4648718A · Sadamitsu et al. · 1987 [cited by applicant]
US 4989079A · Ito · 1991 [cited by applicant]
US 5483339A · Van Aken et al. · 1996 [cited by applicant]
US 6297420B1 · Heincke · 2001 [cited by examiner]
US 6437866B1 · Flynn · 2002 [cited by applicant]
US 6522977B2 · Corrigan et al. · 2003 [cited by applicant]
US 6603550B1 · Flynn et al. · 2003 [cited by applicant]
US 7180524B1 · Axelrod · 2007 [cited by applicant]
US 7738032B2 · Kollias et al. · 2010 [cited by applicant]
US 8182091B2 · Foster · 2012 [cited by applicant]
US 8564778B1 · Igarashi · 2013 [cited by applicant]
US 9019296B1 · Abraham · 2015 [cited by applicant]
US 9064279B1 · Tuan et al. · 2015 [cited by applicant]
US 9519927B1 · Tuan et al. · 2016 [cited by applicant]
US 10045820B2 · Youngquist · 2018 [cited by examiner]
US 10387937B1 · Tuan · 2019 [cited by examiner]
US 20020036778A1 · Wagner et al. · 2002 [cited by applicant]
US 20020059248A1 · Farchione · 2002 [cited by applicant]
US 20020062263A1 · Katou · 2002 [cited by applicant]
US 20020065452A1 · Bazin et al. · 2002 [cited by applicant]
US 20030058261A1 · Challa et al. · 2003 [cited by applicant]
US 20030065578A1 · Peyrelevade et al. · 2003 [cited by applicant]
US 20030065636A1 · Peyrelevade · 2003 [cited by applicant]
US 20030149504A1 · Iwaki et al. · 2003 [cited by applicant]
US 20030151611A1 · Turpin et al. · 2003 [cited by applicant]
US 20030235544A1 · Kishida et al. · 2003 [cited by applicant]
US 20040010333A1 · Baker · 2004 [cited by applicant]
US 20050146531A1 · Rice et al. · 2005 [cited by applicant]
US 20060022994A1 · Hussie · 2006 [cited by applicant]
US 20060066629A1 · Norlander et al. · 2006 [cited by applicant]
US 20060095297A1 · Virik · 2006 [cited by applicant]
US 20060178904A1 · Aghassian et al. · 2006 [cited by applicant]
US 20060179045A1 · Grinsfelder et al. · 2006 [cited by applicant]
US 20060253176A1 · Caruso · 2006 [cited by applicant]
US 20060278589A1 · Peponis · 2006 [cited by examiner]
US 20060289374A1 · Alderman · 2006 [cited by examiner]
US 20070058858A1 · Harville · 2007 [cited by examiner]
US 20070058860A1 · Harville et al. · 2007 [cited by applicant]
US 20070071314A1 · Bhatti et al. · 2007 [cited by applicant]
US 20080003547A1 · Woolfe et al. · 2008 [cited by applicant]
US 20080046410A1 · Lieb · 2008 [cited by applicant]
US 20080079843A1 · Pote et al. · 2008 [cited by applicant]
US 20080194928A1 · Bandic et al. · 2008 [cited by applicant]
US 20080281728A1 · Gomez · 2008 [cited by applicant]
US 20080311061A1 · Heuer · 2008 [cited by applicant]
US 20100020095A1 · Reynolds et al. · 2010 [cited by applicant]
US 20100183528A1 · Maloney et al. · 2010 [cited by applicant]
US 20100185064A1 · Bandic et al. · 2010 [cited by applicant]
US 20100322878A1 · Stella et al. · 2010 [cited by applicant]
US 20110093361A1 · Morales · 2011 [cited by applicant]
US 20110106650A1 · Stone · 2011 [cited by applicant]
US 20110184837A1 · Biro · 2011 [cited by applicant]
US 20110246284A1 · Chaikin et al. · 2011 [cited by applicant]
US 20120045121A1 · Youngman et al. · 2012 [cited by applicant]
US 20120096380A1 · Wagner · 2012 [cited by applicant]
US 20120123759A1 · Cho et al. · 2012 [cited by applicant]
US 20120203668A1 · Howard et al. · 2012 [cited by applicant]
US 20120253224A1 · Mir et al. · 2012 [cited by applicant]
US 20120269743A1 · Smith et al. · 2012 [cited by applicant]
US 20130092805A1 · Funk et al. · 2013 [cited by applicant]
US 20130100008A1 · Marti · 2013 [cited by examiner]
US 20130117137A1 · Klein et al. · 2013 [cited by applicant]
US 20130177137A1 · Eichhorn et al. · 2013 [cited by applicant]
US 20130250322A1 · Kawabata et al. · 2013 [cited by applicant]
US 20130300761A1 · Ahmed · 2013 [cited by applicant]
US 20140323873A1 · Cummins et al. · 2014 [cited by applicant]
US 20150356661A1 · Rousay · 2015 [cited by applicant]
US 20180360709A1 · Rabe · 2018 [cited by examiner]
US 20190038211A1 · Rattner · 2019 [cited by applicant]
US 20190125249A1 · Rattner et al. · 2019 [cited by applicant]
US 20190164210A1 · Kornilov · 2019 [cited by applicant]
US 20190318505A1 · LaSalle et al. · 2019 [cited by applicant]
US 20200056943A1 · Douglas-Sydnor · 2020 [cited by examiner]
US 20200226657A1 · Ji et al. · 2020 [cited by applicant]
US 20210015240A1 · Elfakhri · 2021 [cited by applicant]
US 20220138972A1 · Rattner et al. · 2022 [cited by applicant]
US 20220192506A1 · Lightsey · 2022 [cited by examiner]
US 20230169749A1 · Zhou · 2023 [cited by applicant]
CN 111400606A · 2020 [cited by applicant]
WO 2012065037A1 · 2012 [cited by applicant]
WO 2015126988A1 · 2015 [cited by applicant]
WO 2017181293A1 · 2017 [cited by applicant]
Chanel's new app helps you “try on” the perfect lipstick shade. Whitbread, Louise. The Independent (Online) London: Independent Digital News & Media. (Feb. 22, 2021) (Year: 2021). [cited by examiner]
How Sephora is leveraging AR and AI to transform retail and help customers buy cosmetics. ADN Rayome. TechRepublic, Feb. 2018 (Year: 2018). [cited by examiner]
Alexander, Antoinette, “Virtual Mirror Kiosks Make Beauty Shopping ‘EZ’,” 2009, Drug Store News, Feb. 9, 2009; vol. 31, No. 2, 66 pages. [cited by applicant]
Weisstein, Eric W., “Subset,” Mathworld, retrieved from http://mathworld.wolfram.com/Subset.html, dated May 16, 2013, retrieved from the Internet Archive on Jul. 12, 2015, 1 page. [cited by applicant]
Hastie, Trevor, et al., “The Elements of Statistical Learning Data Mining, Inference, and Prediction,” Springer Series in Statistics, 2nd ed., Chapters 14.3-14.6 and 16, Jan. 13, 2017, retrieved from https://hastie.su.d… [cited by applicant]
Goodfellow, Ian, et al., “Deep Learning,” MIT Press book, 2016, Part I, Chapters 7 and 10, Part III, Chapters 14 and 20, retrieved from https://www.deeplearningbook.org/ 196 pages,. [cited by applicant]
PCT International Search Report and the Written Opinion of the International Searching Authority for International Application PCT/US2022/043155 dated Mar. 26, 2023, 14 pgs. [cited by applicant]
Molnar, Christoph, “Interpretable Machine Learning—A Guide for Making Black Box Models Explainable,” Aug. 26, 2021, Chapters 4.1 and 5, retrieved from https://web.archive.org/web/20210828033303/https://christophm.github… [cited by applicant]