IP Library Granted Patent US 12,566,489
Granted Patent B2
US 12,566,489 · App. 18/821,659 · Granted Mar 3, 2026

Monitoring a user of a head-wearable electronic device

Inventors: Joel S. Armstrong-Muntner (San Carlos, CA); Nicholas P. Allec (Santa Cruz, CA); James E. Stark (New York, NY)
Assignee: Apple Inc.
G06F3/011G02B27/017G06F1/163G06F3/012G06F3/017G06F3/0304G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,566,489
App. No.
18/821,659
Granted
Mar 3, 2026
Kind
B2
Abstract

Systems, methods, and computer-readable media for monitoring a user of a head-wearable electronic device with multiple light-sensing assemblies. For instance, methods and/or related systems or computer readable media programmed to perform the methods, directed to the selection of light sensing assemblies and/or light sensor data based on the detected saturation of the light-sensing assemblies and/or components.

Claims (52)

1 . A method of detecting gestures of a user at an electronic device including

one or more light-sensing assemblies, the method comprising:

collecting light sensor data from one or more light-sensing components of a first light-sensing assembly during a sensing period; and

detecting a level of ambient light received by one or more light-sensing components of the first light-sensing assembly;

in accordance with a determination that one or more first criteria are satisfied, including a criterion that is satisfied when a level of ambient light received by each of the one or more light-sensing components of the first light-sensing assembly is below a first threshold of saturation, selecting each channel of light sensor data corresponding to each of the one or more light-sensing components; and

in accordance with a determination that the one or more first criteria are not satisfied, selecting one or more channels of light sensor data corresponding to a portion of the one or more light-sensing components of the first light-sensing assembly or forgoing selecting a channel of light sensor data.

2 . The method of claim 1 , further comprising:

in accordance with a determination that the one or more first criteria are not satisfied:

determining whether each of the one or more light-sensing components satisfies one or more second criteria, including a criterion that is satisfied when the level of ambient light received by each of the one or more light-sensing components of the first light-sensing assembly is below a second threshold of saturation.

3 . The method of claim 2 , wherein the second threshold of saturation is greater than the first threshold of saturation.

4 . The method of claim 2 , further comprising:

in accordance with a determination that each of the one or more light-sensing components satisfies the one or more second criteria:

selecting the one or more channels of light sensor data corresponding to a first subset of the one or more light-sensing components that satisfy one or more third criteria, including a criterion that is satisfied when the level of ambient light received by each of the first subset of the one or more light-sensing components is below a third threshold of saturation.

5 . The method of claim 4 , wherein the third threshold of saturation is greater than the second threshold of saturation.

6 . The method of claim 4 , further comprising:

identifying a second subset of the one or more light-sensing components, wherein each of the second subset of the one or more light-sensing components fails to satisfy the one or more third criteria; and

in accordance with a determination that one or more of the second subset of the one or more light-sensing components fail to satisfy one or more fourth criteria, including a criterion that is satisfied when the level of ambient light received by each of one or more each of light-sensing components of the second subset is below a fourth threshold of saturation, terminating the sensing period.

7 . The method of claim 6 , wherein in accordance with a determination that one or more of the second subset of the one or more light-sensing components satisfy the one or more fourth criteria, continuing the sensing period.

8 . The method of claim 6 , wherein the fourth threshold of saturation is greater than the third threshold of saturation.

9 . An electronic device, comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

collecting light sensor data from one or more light-sensing components of a first light-sensing assembly during a sensing period; and

detecting a level of ambient light received by one or more light-sensing components of the first light-sensing assembly;

in accordance with a determination that one or more first criteria are satisfied, including a criterion that is satisfied when a level of ambient light received by each of the one or more light-sensing components of the first light-sensing assembly is below a first threshold of saturation, selecting each channel of light sensor data corresponding to each of the one or more light-sensing components; and

in accordance with a determination that the one or more first criteria are not satisfied, selecting one or more channels of light sensor data corresponding to a portion of the one or more light-sensing components of the first light-sensing assembly or forgoing selecting a channel of light sensor data.

10 . The electronic device of claim 9 , the one or more programs further comprising instructions for:

in accordance with a determination that the one or more first criteria are not satisfied: determining whether each of the one or more light-sensing components satisfies one or more second criteria, including a criterion that is satisfied when the level of ambient light received by each of the one or more light-sensing components of the first light-sensing assembly is below a second threshold of saturation.

11 . The electronic device of claim 10 , wherein the second threshold of saturation is greater than the first threshold of saturation.

12 . The electronic device of claim 10 , the one or more programs further comprising instructions for:

in accordance with a determination that each of the one or more light-sensing components satisfies the one or more second criteria:

selecting the one or more channels of light sensor data corresponding to a first subset of the one or more light-sensing components that satisfy one or more third criteria, including a criterion that is satisfied when the level of ambient light received by each of the first subset of the one or more light-sensing components is below a third threshold of saturation.

13 . The electronic device of claim 12 , wherein the third threshold of saturation is greater than the second threshold of saturation.

14 . The electronic device of claim 12 , the one or more programs further comprising instructions for:

identifying a second subset of the one or more light-sensing components, wherein each of the second subset of the one or more light-sensing components fails to satisfy the one or more third criteria; and

in accordance with a determination that one or more of the second subset of the one or more light-sensing components fail to satisfy one or more fourth criteria, including a criterion that is satisfied when the level of ambient light received by each of one or more each of light-sensing components of the second subset is below a fourth threshold of saturation, terminating the sensing period.

15 . A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:

collect light sensor data from one or more light-sensing components of a first light-sensing assembly during a sensing period; and

detect a level of ambient light received by one or more light-sensing components of the first light-sensing assembly;

in accordance with a determination that one or more first criteria are satisfied, including a criterion that is satisfied when a level of ambient light received by each of the one or more light-sensing components of the first light-sensing assembly is below a first threshold of saturation, select each channel of light sensor data corresponding to each of the one or more light-sensing components; and

in accordance with a determination that the one or more first criteria are not satisfied, select one or more channels of light sensor data corresponding to a portion of the one or more light-sensing components of the first light-sensing assembly or forgoing selecting a channel of light sensor data.

16 . The non-transitory computer readable storage medium of claim 15 , the one or more programs further comprising instructions which cause the electronic device to:

in accordance with a determination that the one or more first criteria are not satisfied: determine whether each of the one or more light-sensing components satisfies one or more second criteria, including a criterion that is satisfied when the level of ambient light received by each of the one or more light-sensing components of the first light-sensing assembly is below a second threshold of saturation.

17 . The non-transitory computer readable storage medium of claim 16 , wherein the second threshold of saturation is greater than the first threshold of saturation.

18 . The non-transitory computer readable storage medium of claim 16 , the one or more programs further comprising instructions which cause the electronic device to:

in accordance with a determination that each of the one or more light-sensing components satisfies the one or more second criteria:

select the one or more channels of light sensor data corresponding to a first subset of the one or more light-sensing components that satisfy one or more third criteria, including a criterion that is satisfied when the level of ambient light received by each of the first subset of the one or more light-sensing components is below a third threshold of saturation.

19 . The non-transitory computer readable storage medium of claim 18 , wherein the third threshold of saturation is greater than the second threshold of saturation.

20 . The non-transitory computer readable storage medium of claim 18 , the one or more programs further comprising instructions which cause the electronic device to:

identify a second subset of the one or more light-sensing components, wherein each of the second subset of the one or more light-sensing components fails to satisfy the one or more third criteria; and

in accordance with a determination that one or more of the second subset of the one or more light-sensing components fail to satisfy one or more fourth criteria, including a criterion that is satisfied when the level of ambient light received by each of one or more each of light-sensing components of the second subset is below a fourth threshold of saturation, terminate the sensing period.

Continuity (7)
Continuation 18059347 · Nov 28, 2022
Continuation 17938596 · Oct 6, 2022
Division 17073936 · Oct 19, 2020
Continuation 16144485 · Sep 27, 2018
Provisional Application 62718937 · Aug 14, 2018
Provisional Application 62565272 · Sep 29, 2017
Related Publication 20240419239A1 · Dec 19, 2024
References Cited (49)
US 7221374B2 · Dixon · 2007 [cited by examiner]
US 7515054B2 · Torch · 2009 [cited by applicant]
US 8154478B2 · Park · 2012 [cited by examiner]
US 8305370B2 · Song · 2012 [cited by examiner]
US 8432340B2 · Okita · 2013 [cited by examiner]
US 8686981B2 · Barnhoefer · 2014 [cited by examiner]
US 8884939B2 · Barnhoefer · 2014 [cited by examiner]
US 9119261B2 · Barnhoefer · 2015 [cited by examiner]
US 9171198B1 · Raffle et al. · 2015 [cited by applicant]
US 9355314B2 · Yang · 2016 [cited by examiner]
US 9542016B2 · Armstrong-muntner · 2017 [cited by applicant]
US 9807291B1 · Tome et al. · 2017 [cited by applicant]
US 10121063B2 · Von Und Zu Liechtenstein · 2018 [cited by applicant]
US 10252058B1 · Fuerst · 2019 [cited by applicant]
US 10509466B1 · Miller et al. · 2019 [cited by applicant]
US 10592002B2 · Iyer · 2020 [cited by examiner]
US 10747327B2 · Iyer · 2020 [cited by examiner]
US 20130050258A1 · Liu et al. · 2013 [cited by applicant]
US 20130289886A1 · Ricks · 2013 [cited by applicant]
US 20140354514A1 · Aronsson · 2014 [cited by applicant]
US 20150058615A1 · Lee · 2015 [cited by applicant]
US 20150087924A1 · Li et al. · 2015 [cited by applicant]
US 20150139509A1 · Yang · 2015 [cited by applicant]
US 20160187652A1 · Fujimaki et al. · 2016 [cited by applicant]
US 20170090567A1 · Allec et al. · 2017 [cited by applicant]
US 20170111723A1 · Boesen · 2017 [cited by applicant]
US 20170323485A1 · Samec et al. · 2017 [cited by applicant]
US 20170371417A1 · Iyer et al. · 2017 [cited by applicant]
US 20180061128A1 · Cabanier et al. · 2018 [cited by applicant]
US 20180137395A1 · Han et al. · 2018 [cited by applicant]
US 20190018485A1 · Aleem et al. · 2019 [cited by applicant]
US 20190101977A1 · Armstrong-muntner et al. · 2019 [cited by applicant]
US 20190101984A1 · Talati et al. · 2019 [cited by applicant]
US 20190313073A1 · Patel · 2019 [cited by examiner]
US 20190350472A1 · Zhovnirovsky · 2019 [cited by applicant]
US 20190384408A1 · Iyer · 2019 [cited by examiner]
US 20200174262A1 · Godar et al. · 2020 [cited by applicant]
US 20210034145A1 · Armstrong-muntner et al. · 2021 [cited by applicant]
US 20230297163A1 · Armstrong-muntner et al. · 2023 [cited by applicant]
Final Office Action received for U.S. Appl. No. 17/073,936, mailed on Jun. 6, 2022, 22 pages. [cited by applicant]
Final Office Action received for U.S. Appl. No. 18/059,347, mailed on Jan. 19, 2024, 7 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 16/144,485, mailed on Feb. 7, 2020, 15 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 17/073,936, mailed on Dec. 15, 2021, 16 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 18/059,347, mailed on Aug. 17, 2023, 17 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 16/144,485, mailed on Jun. 15, 2020, 15 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 18/059,347, mailed on May 3, 2024, 10 pages. [cited by applicant]
Restriction Requirement received for U.S. Appl. No. 16/144,485, mailed on Nov. 15, 2019, 9 pages. [cited by applicant]
Goudreau, Jenna, “7 Signs Someone's Not Listening To You”, Business Insider, Available online at: <https://www.businessinsider.com/signs-someone-is-not-listening-2014-6>, Jun. 11, 2014, 3 pages. [cited by applicant]
Quora, “Does a blank facial expression still count as a nonverbal cue?”, Available online at: <https://www.quora.com/Does-a-blank-facial-expression-still-count-as-a-nonverbal-cue>, Nov. 19, 2017, 4 pages. [cited by applicant]