IP Library Granted Patent US 12,437,869
Granted Patent B2
US 12,437,869 · App. 17/580,566 · Granted Oct 7, 2025

Headset integrated into healthcare platform

Inventors: Robert Konrad (San Francisco, CA); Kevin Boyle (San Francisco, CA); Nitish Padmanaban (Menlo Park, CA); Gordon Wetzstein (Palo Alto, CA)
Assignee: Sesame AI, Inc.
G16H40/67G02B27/0093G02B27/017G16H10/60G16H50/20G02B2027/0138G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,437,869
App. No.
17/580,566
Granted
Oct 7, 2025
Kind
B2
Abstract

Embodiments are related to a headset integrated into a healthcare platform. The headset comprises one or more sensors embedded into a frame of the headset, a controller coupled to the one or more sensors, and a transceiver coupled to the controller. The one or more sensors capture health information data for a user wearing the headset. The controller pre-processes at least a portion of the captured health information data to generate a pre-processed portion of the health information data. The transceiver communicates the health information data and the pre-processed portion of health information data to an intermediate device communicatively coupled to the headset. The intermediate device processes at least one of the health information data and the pre-processed portion of health information data to generate processed health information data for a health-related diagnostic of the user.

Claims (76)

1. A headset comprising:

a plurality of sensors embedded into a frame of the headset and configured to capture health information data for a user wearing the headset, the plurality of sensors including an acoustic sensor configured to capture auditory information associated with the user and a plurality of gaze sensors, each of the plurality of gaze sensors comprising:

a light emitter configured to emit tracking light to an eye of the user, and

a detector configured to:

detect a signal of a plurality of signals related to a version of the tracking light reflected from at least one surface of the eye, wherein the plurality of signals detected by the plurality of gaze sensors comprise eye-related data, and

capture an image of the eye based on the version of the tracking light reflected from the at least one surface of the eye;

a pair of lenses including a focus-tunable lens capable of having a focal distance dynamically adjusted based on the image of the eye captured by the detector;

a controller coupled to the plurality of sensors, the controller configured to:

process, using a natural language processing algorithm, the auditory information to classify the user as exhibiting a language disorder,

extract, from the eye-related data, information about dilations of pupils of eyes of the user, and

determine, based on the extracted information about dilations of pupils, an indication of a possible neurological damage for the user; and

a transceiver coupled to the controller, the transceiver configured to communicate, to a wireless intermediate device via a wireless connection, user's data including the health information data, an indication of the user exhibiting the language disorder, and the indication of the possible neurological damage,

wherein the wireless intermediate device processes the user's data received from the headset to generate processed user's data for a health-related diagnostic of the user and communicates, via a second wireless connection, a portion of the user's data and a portion of the processed user's data to a server platform for further processing.

2. The headset of claim 1 , wherein the health information data comprise at least one of: data related to a skin of the user, data related to an orientation of a head of the user, data related to a brain activity for the user, data related to a tracked activity of the user, data related to a breathing of the user, and data related to an environment of the headset.

3. The headset of claim 1 , wherein the controller is further configured to extract one or more features related to the user from the captured health information data.

4. The headset of claim 3 , wherein the extracted one or more features comprise at least one of: one or more features of at least one eye of the user, and one or more features related to a mouth of the user.

5. The headset of claim 4 , wherein:

the one or more features of the at least one eye comprise information about eyelid statistics of the user over time; and

the controller is further configured to process the information about the eyelid statistics for the health-related diagnostic of the user.

6. The headset of claim 1 , wherein the transceiver is further configured to communicate, to the intermediate device, the health information data at a first frequency that is less than a second frequency at which a pre-processed portion of the health information data would be communicated.

7. The headset of claim 1 , wherein:

the controller is further configured to determine a respective rank for each type of data of the captured health information data; and

the transceiver is further configured to communicate each type of data to the intermediate device in accordance with the respective rank.

8. The headset of claim 1 , wherein the processed user's data is used by a healthcare application running on the intermediate device for the health-related diagnostic.

9. The headset of claim 1 , wherein the intermediate device communicates the processed user's data to one or more users of the intermediate device for providing information about the health-related diagnostic to the one or more users.

10. The headset of claim 1 , wherein the intermediate device further communicates the portion of the processed user's data to one or more application devices for at least one of further processing and utilization.

11. The headset of claim 1 , wherein the server platform extracts at least one of statistics and features from the portion of the processed user's data and provides an access to the at least one of extracted statistics and features for at least one of one or more partner services and one or more application devices.

12. A method comprising:

capturing, by a plurality of sensors embedded into a frame of a headset, health information data for a user wearing the headset, the plurality of sensors including an acoustic sensor configured to capture auditory information associated with the user and a plurality of gaze sensors, each of the plurality of gaze sensors comprising:

a light emitter configured to emit tracking light to an eye of the user, and

a detector configured to:

detect a signal of a plurality of signals related to a version of the tracking light reflected from at least one surface of the eye, wherein the plurality of signals detected by the plurality of gaze sensors comprise eye-related data, and

capture an image of the eye based on the version of the tracking light reflected from the at least one surface of the eye;

dynamically adjusting, based on the image of the eye captured by the detector, a focal distance of a focus-tunable lens of a pair of lenses of the headset;

processing, by a controller coupled to the plurality of sensors and using a natural language processing algorithm, the auditory information to classify the user as exhibiting a language disorder;

extracting, by the controller and from the eye-related data, information about dilations of pupils of eyes of the user;

determining, by the controller and based on the extracted information about dilations of pupils, an indication of a possible neurological damage for the user; and

communicating, by a transceiver coupled to the controller to a wireless intermediate device via a wireless connection, user's data including the health information data, an indication of the user exhibiting the language disorder, and the indication of the possible neurological damage for processing the user's data received from the headset at the intermediate device to generate processed user's data for a health-related diagnostic of the user,

wherein the wireless intermediate device communicates, via a second wireless connection, a portion of the user's data and a portion of the processed user's data to a server platform for further processing.

13. The method of claim 12 , wherein capturing the health information data comprises:

capturing at least one of data related to a skin of the user, data related to an orientation of a head of the user, data related to a brain activity for the user, data related to a tracked activity of the user, data related to a breathing of the user, and data related to an environment of the headset.

14. The method of claim 12 , further comprising:

extracting, by the controller, one or more features related to the user from the captured health information data,

wherein the extracted one or more features comprise at least one of one or more features of at least one eye of the user, and one or more features related to a mouth of the user.

15. The method of claim 14 , further comprising:

extracting, by the controller, the one or more features of the at least one eye related to information about eyelid statistics of the user over time; and

processing, by the controller, the information about the eyelid statistics for the health-related diagnostic of the user.

16. The method of claim 12 , further comprising:

communicating, by the transceiver to the intermediate device, the health information data at a first frequency that is less than a second frequency at which a pre-processed portion of the health information data would be communicated.

17. The method of claim 12 , further comprising:

determining, by the controller, a respective rank for each type of data of the captured health information data; and

communicating, by the transceiver, each type of data to the intermediate device in accordance with the respective rank.

18. The method of claim 12 , wherein:

the portion of the processed user's data is further communicated to one or more application devices for at least one of further processing and utilization; and

the server platform extracts at least one of statistics and features from the portion of the processed user's data and provides an access to the at least one of extracted statistics and features for at least one of one or more partner services and the one or more application devices.

19. A system comprising:

a headset including:

a plurality of sensors embedded into a frame of the headset and configured to capture health information data for a user wearing the headset, the plurality of sensors including an acoustic sensor configured to capture auditory information associated with the user and a plurality of gaze sensors, each of the plurality of gaze sensors comprising:

a light emitter configured to emit tracking light to an eye of the user, and

a detector configured to:

detect a signal of a plurality of signals related to a version of the tracking light reflected from at least one surface of the eye, wherein the plurality of signals detected by the plurality of gaze sensors comprise eye-related data, and

capture an image of the eye based on the version of the tracking light reflected from the at least one surface of the eye,

a pair of lenses including a focus-tunable lens capable of having a focal distance dynamically adjusted based on the image of the eye captured by the detector,

a controller coupled to the plurality of sensors, the controller configured to:

process, using a natural language processing algorithm, the auditory information to classify the user as exhibiting a language disorder,

extract, from the eye-related data, information about dilations of pupils of eyes of the user, and

determine, based on the extracted information about dilations of pupils, an indication of a possible neurological damage for the user, and

a transceiver coupled to the controller, the transceiver configured to communicate user's data including the health information data, an indication of the user exhibiting the language disorder, and the indication of the possible neurological damage outside of the headset; and

a wireless intermediate device communicatively coupled to the headset via a wireless connection, the wireless intermediate device including:

a second transceiver configured to receive the user's data via the wireless connection, and

a second controller configured to process the received user's data to generate processed user's data for a health-related diagnostic of the user,

wherein the second transceiver is further configured to communicate, via a second wireless connection, a portion of the user's data and a portion of the processed user's data to a server platform for further processing.

20. The system of claim 19 , wherein:

the controller is further configured to extract one or more features related to the user from the captured health information data;

the extracted one or more features comprise at least one of: one or more features of at least one eye of the user, and one or more features related to a mouth of the user; and

the one or more features of the at least one eye comprise information about eyelid statistics of the user over time.

Assignments (2)
MERGER Recorded Aug 30, 2024
From: ZINN LABS, INC.
To: SESAME AI, INC.
Reel/Frame 068455/0209 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 24, 2022
From: KONRAD, ROBERT; BOYLE, KEVIN; PADMANABAN, NITISH; WETZSTEIN, GORDON
To: ZINN LABS, INC.
Reel/Frame 058745/0205 →
Continuity (4)
Provisional Application 63140701 · Jan 22, 2021
Provisional Application 63153176 · Feb 24, 2021
Provisional Application 63173742 · Apr 12, 2021
Related Publication 20220238220A1 · Jul 28, 2022
References Cited (71)
US 7558622B2 · Tran · 2009 [cited by examiner]
US 10228565B1 · Saarikko · 2019 [cited by examiner]
US 10395111B2 · Konttori et al. · 2019 [cited by applicant]
US 10437327B2 · Sengelaub · 2019 [cited by examiner]
US 10474229B1 · Gollier · 2019 [cited by examiner]
US 10610111B1 · Tran · 2020 [cited by examiner]
US 10638938B1 · Tzvieli · 2020 [cited by examiner]
US 11217033B1 · Morgan · 2022 [cited by examiner]
US 11269406B1 · Sztuk et al. · 2022 [cited by applicant]
US 11617559B2 · Samec · 2023 [cited by examiner]
US 20010043163A1 · Waldern et al. · 2001 [cited by applicant]
US 20050182654A1 · Abolfathi · 2005 [cited by examiner]
US 20050203366A1 · Donoghue · 2005 [cited by examiner]
US 20080246629A1 · Tsui · 2008 [cited by examiner]
US 20110288379A1 · Wu · 2011 [cited by applicant]
US 20120019529A1 · Kimpe · 2012 [cited by applicant]
US 20120116772A1 · Jones · 2012 [cited by examiner]
US 20120330109A1 · Tran · 2012 [cited by examiner]
US 20130013331A1 · Horseman · 2013 [cited by examiner]
US 20150142462A1 · Vaidya · 2015 [cited by examiner]
US 20150278474A1 · Stueckemann · 2015 [cited by examiner]
US 20150359467A1 · Tran · 2015 [cited by examiner]
US 20160077585A1 · Mizuhara · 2016 [cited by applicant]
US 20160195924A1 · Weber et al. · 2016 [cited by applicant]
US 20160292850A1 · Perez et al. · 2016 [cited by applicant]
US 20160331255A1 · Cheatham, III · 2016 [cited by examiner]
US 20170007167A1 · Kostic · 2017 [cited by examiner]
US 20170039703A1 · Wang · 2017 [cited by examiner]
US 20170195637A1 · Kusens · 2017 [cited by examiner]
US 20170235931A1 · Publicover · 2017 [cited by examiner]
US 20170365101A1 · Samec · 2017 [cited by examiner]
US 20180039386A1 · Nishiyama et al. · 2018 [cited by applicant]
US 20180054568A1 · Sugawara · 2018 [cited by applicant]
US 20190046037A1 · Ramesh · 2019 [cited by examiner]
US 20190103143A1 · Hasbun · 2019 [cited by examiner]
US 20190327567A1 · Blum et al. · 2019 [cited by applicant]
US 20190354173A1 · Young et al. · 2019 [cited by applicant]
US 20190354174A1 · Young et al. · 2019 [cited by applicant]
US 20190370926A1 · Hagland · 2019 [cited by applicant]
US 20200005026A1 · Andersen · 2020 [cited by examiner]
US 20200037942A1 · Howard · 2020 [cited by examiner]
US 20200174255A1 · Hollands et al. · 2020 [cited by applicant]
US 20200211555A1 · Khaleghi · 2020 [cited by examiner]
US 20200350075A1 · Noch · 2020 [cited by examiner]
US 20210169417A1 · Burton · 2021 [cited by examiner]
US 20210247610A1 · Bhakta et al. · 2021 [cited by applicant]
US 20210341998A1 · Sun · 2021 [cited by applicant]
US 20220004009A1 · Myhre et al. · 2022 [cited by applicant]
US 20220005798A1 · Qi et al. · 2022 [cited by applicant]
US 20220007936A1 · Yang · 2022 [cited by examiner]
US 20220011496A1 · Bhakta et al. · 2022 [cited by applicant]
US 20220100269A1 · Eash et al. · 2022 [cited by applicant]
US 20220238220A1 · Konrad et al. · 2022 [cited by applicant]
US 20220261076A1 · Choi et al. · 2022 [cited by applicant]
US 20240115202A1 · Tran · 2024 [cited by examiner]
RU 2724442C1 · 2020 [cited by applicant]
WO WO2020224041A1 · 2020 [cited by applicant]
Li et. al., Computer Vision for Brain Disorders Based Primarily on Ocular Responses, Apr. 20, 2021, Front. Neurol., vol. 12—2021 (Year: 2021). [cited by examiner]
Aksit, K. et al. “Gaze-Sensing LEDs for Head Mounted Displays.” arXiv Preprint arXiv:2003.08499v1, Mar. 20, 2020, pp. 1-14. [cited by applicant]
Angelopoulos, A. N. et al. “Event Based, Near Eye Gaze Tracking Beyond 10,000Hz.” IEEE Transactions on Visualization and Computer Graphics, Feb. 5, 2021, pp. 1-12. [cited by applicant]
Borsato, F. H. et al. “A Fast and Accurate Eye Tracker Using Stroboscopic Differential Lighting.” Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops, Dec. 2015, pp. 110-118. [cited by applicant]
Katrychuk, D. et al. “Power-Efficient and Shift-Robust Eye-Tracking Sensor for Portable VR Headsets.” Proceedings of the 11 [cited by applicant]
Li, T. et al. “Battery-Free Eye Tracker on Glasses.” Proceedings of the 24 [cited by applicant]
Mastrangelo, A. S. et al. “A Low-Profile Digital Eye-Tracking Oculometer for Smart Eyeglasses.” 11 [cited by applicant]
Rigas, I. et al. “Photosensor Oculography: Survey and Parametric Analysis of Designs Using Model-Based Simulation.” IEEE Transactions on Human-Machine Systems, vol. 48, No. 6, Dec. 2018, pp. 1-12. [cited by applicant]
TobiiPro. “Dark and Bright Pupil Tracking.” TobiiPro.com, 2022, 5 pages, [Online] [Retrieved Apr. 25, 2022], Retrieved from the Internet <URL:https://www.tobiipro.com/learn-and-support/learn/eye-tracking-essentials/what… [cited by applicant]
Padmanaban, N. et al., “Autofocals: Evaluating gaze-contingent eyeglasses for presbyopes,” Science Advances, vol. 5, Jun. 28, 2019, pp. 1-7. [cited by applicant]
Padmanaban, N., “Enabling Gaze-Contingent Accommodation in Presbyopia Correction and Near-Eye Displays,” A Dissertation Submitted to the Department of Electrical Engineering and the Committee on Graduate Studies of Stan… [cited by applicant]
PCT International Search Report and Written Opinion, PCT Application No. PCT/US2022/013193, Apr. 21, 2022, 16 pages. [cited by applicant]
PCT International Search Report and Written Opinion, PCT Application No. PCT/US2022/013195, Apr. 26, 2022, 14 pages. [cited by applicant]
United States Office Action, U.S. Appl. No. 17/580,568, filed Sep. 23, 2022, eight pages. [cited by applicant]