IP Library Granted Patent US 12,315,640
Granted Patent B2
US 12,315,640 · App. 17/844,614 · Granted May 27, 2025

Atmospheric mirroring and dynamically varying three-dimensional assistant addison interface for interior environments

Inventors: Anthony Dohrmann (El Paso, TX); Roberto Abel Salcido (El Paso, TX); David W. Keeley (Frisco, TX); Samuel Blake (Las Cruces, NM); Madison Anne Markle (Las Cruces, NM); Sierra Danielle Guerrero (Las Cruces, NM); Taylor Allen Bunker (Las Cruces, NM); Judah Tveito (Las Cruces, NM)
Assignee: Electronic Caregiver, Inc.
G16H50/30G10L15/22G16H40/60G16H50/20G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,315,640
App. No.
17/844,614
Filed
Jun 20, 2022
Granted
May 27, 2025
Kind
B2
Art Unit
3683
USPC
600/195
Abstract

Exemplary embodiments include an intelligent secure networked health messaging system configured by at least one processor to execute instructions stored in memory, the system including a data retention system and a health analytics system, the health analytics system performing asynchronous processing with a patient's computing device and the health analytics system communicatively coupled to a deep neural network, a web services layer providing access to the data retention and the health analytics system, a batching service, wherein an application server layer transmits a request to the web services layer for data, the request processed by the batching service transparently to the patient, the request processed by the batching service transparently to the patient such that the patient can continue to use a patient facing application without disruption, and the patient-facing application having an audio sensor and a computer video sensor.

Claims (34)

1. An intelligent secure networked health messaging system configured by at least one processor to execute instructions stored in memory, the system comprising:

a data retention system and a health analytics system, the health analytics system performing asynchronous processing with a patient's computing device and the health analytics system communicatively coupled to a deep neural network, the data retention system and the health analytics system coupled to the deep neural network configured as an isolated sub-system in secure isolation from a remainder of the intelligent secure networked health messaging system via one of a security protocol or layer;

a web services layer providing access to the data retention and the health analytics system;

a batching service, wherein an application server layer transmits a request to the web services layer for data, the request processed by the batching service asynchronously to a patient-facing application to reduce latency in data display and storage and to update operations of the patient-facing application;

the application server layer including a data corridor established between the application server layer and the patient's computing device that:

provides the patient-facing application that accesses the data retention system, the health analytics system, and the deep neural network through the web services layer;

performs processing based on patient interaction with the patient-facing application, the patient-facing application configured to execute instructions including transmitting an interactive conversational patient interface to the patient's computing device;

the deep neural network configured to:

receive a first input at an input layer;

process the first input at one or more hidden layers;

generate a first output;

transmit the first output to an output layer;

provide the first output to the patient-facing application; and

provide the first output to the interactive conversational patient interface;

the patient-facing application with the interactive conversational patient interface converting the first output received by the patient's computing device into an audio file using a cloud-based text-to-speech application capable of being integrated into a web browser based avatar, the avatar being displayed on a display screen within the web browser of the patient's computing device as a three-dimensional electronic image of a human caregiver for a human patient, further comprising the three-dimensional electronic image of the human caregiver providing step-by-step verbal healthcare instructions to the human patient, monitoring a response from the human patient, and providing healthcare advice to the human patient based on the first output.

2. The intelligent secure networked health messaging system of claim 1 , further comprising the first output generating a first outcome.

3. The intelligent secure networked health messaging system of claim 2 , further comprising the first outcome being transmitted to the input layer, processing the first outcome by the one or more hidden layers, generating a second output, transmitting the second output to the output layer, providing the second output to the patient-facing application and the second output generating a second outcome.

4. The intelligent secure networked health messaging system of claim 3 , further comprising the second outcome being transmitted to the input layer.

5. The intelligent secure networked health messaging system of claim 1 , further comprising the first output including any of a clinically relevant care plan, a reminder, an alert, or a survey.

6. The intelligent secure networked health messaging system of claim 1 , further comprising an outcome including any of a biometric parameter, a biometric parameter out of a predetermined threshold, a response to a survey, medication compliance information, an indicator of daily activity, an indicator of mood, or an indicator of stress.

7. The intelligent secure networked health messaging system of claim 1 , further comprising the processing by the one or more hidden layers including using voice, speech, and computer video inputs to analyze signs of changes in health and behavioral status including but not limited to stress, anger, change in speech cadence, slurred speech or coughing.

8. The intelligent secure networked health messaging system of claim 7 , further comprising the processing determining changes in the health and behavioral status including but not limited to anger, substance use, lack of sleep, stress, early onset of dementia or Alzheimer's disease, an adverse reaction to a medication, a stroke, Parkinson's disease, an increased risk of falling, or a lack of balance.

9. The intelligent secure networked health messaging system of claim 1 , further comprising the interactive conversational patient interface configured to mirror an interior environment.

10. The intelligent secure networked health messaging system of claim 9 , the mirrored interior environment including a realistic depiction of a fireplace that turns on when a temperature is below a certain threshold.

11. The intelligent secure networked health messaging system of claim 9 , the mirrored interior environment including a depiction of the patient's favorite color on an item in the patient's home.

12. The intelligent secure networked health messaging system of claim 9 , the mirrored interior environment including a depiction of the patient's favorite art style on an item in the patient's home.

13. The intelligent secure networked health messaging system of claim 9 , the mirrored interior environment including a depiction of holiday and religious celebration items in the patient's home.

14. The intelligent secure networked health messaging system of claim 9 , the mirrored interior environment including a depiction of the patient's favorite animals or pets in the patient's home.

15. The intelligent secure networked health messaging system of claim 9 , the mirrored interior environment including a depiction of interactable objects that respond when touched in the patient's home.

16. The intelligent secure networked health messaging system of claim 15 , the mirrored interior environment including a depiction of interactable objects including any of a piano, radio, bird feeder, plant, animal, wind chime, teacup, or vase that responds when touched in the patient's home.

17. The intelligent secure networked health messaging system of claim 15 , the mirrored interior environment including a depiction of interactable objects including a book that can be opened and read via touch or voice.

18. The intelligent secure networked health messaging system of claim 9 , the mirrored interior environment including a depiction of a patient's hobby in the patient's home.

19. The intelligent secure networked health messaging system of claim 18 , the mirrored interior environment including a depiction of the patient's hobby in the patient's home, the hobby being skiing.

20. The intelligent secure networked health messaging system of claim 18 , the mirrored interior environment including a depiction of the patient's hobby in the patient's home, the hobby being snowboarding.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 30, 2022
From: DOHRMANN, ANTHONY; SALCIDO, ROBERTO ABEL; BLAKE, SAMUEL; MARKLE, MADISON ANNE; GUERRERO, SIERRA DANIELLE; BUNKER, TAYLOR ALLEN; KEELEY, DAVID W.; TVEITO, JUDAH
To: ELECTRONIC CAREGIVER, INC.
Reel/Frame 060942/0487 →
Continuity (10)
Continuation In Part 17735750 · May 3, 2022
Continuation In Part 17693151 · Mar 11, 2022
Continuation In Part 17013357 · Sep 4, 2020
Continuation In Part 16169760 · Oct 24, 2018
Continuation 15530185 · Dec 9, 2016
Provisional Application 63213625 · Jun 22, 2021
Provisional Application 63184060 · May 4, 2021
Provisional Application 62618550 · Jan 17, 2018
Provisional Application 62386768 · Dec 11, 2015
Related Publication 20220319713A1 · Oct 6, 2022
References Cited (132)
US 7612681B2 · Azzaro et al. · 2009 [cited by applicant]
US 7971141B1 · Quinn et al. · 2011 [cited by applicant]
US 8206325B1 · Najafi et al. · 2012 [cited by applicant]
US 8771206B2 · Gettelman et al. · 2014 [cited by applicant]
US 9072929B1 · Rush et al. · 2015 [cited by applicant]
US 9972187B1 · Srinivasan et al. · 2018 [cited by applicant]
US 10387963B1 · Leise et al. · 2019 [cited by applicant]
US 10417388B2 · Han et al. · 2019 [cited by applicant]
US 10628635B1 · Carpenter, II et al. · 2020 [cited by applicant]
US 10761691B2 · Anzures et al. · 2020 [cited by applicant]
US 10813572B2 · Dohrmann et al. · 2020 [cited by applicant]
US 10943407B1 · Morgan et al. · 2021 [cited by applicant]
US 10998101B1 · Tran et al. · 2021 [cited by applicant]
US 11837341B1 · Chandra S R · 2023 [cited by examiner]
US 12009083B2 · Keeley et al. · 2024 [cited by applicant]
US 12011259B2 · Dohrmann et al. · 2024 [cited by applicant]
US 12265900B2 · Dohrmann et al. · 2025 [cited by applicant]
US 20020062342A1 · Sidles · 2002 [cited by applicant]
US 20040109470A1 · Derechin et al. · 2004 [cited by applicant]
US 20040147817A1 · Dewing · 2004 [cited by examiner]
US 20040189708A1 · Larcheveque · 2004 [cited by applicant]
US 20050035862A1 · Wildman et al. · 2005 [cited by applicant]
US 20070032929A1 · Yoshioka · 2007 [cited by applicant]
US 20080010293A1 · Zpevak et al. · 2008 [cited by applicant]
US 20080186189A1 · Azzaro et al. · 2008 [cited by applicant]
US 20090030945A1 · Miller et al. · 2009 [cited by applicant]
US 20090094285A1 · Mackle et al. · 2009 [cited by applicant]
US 20100124737A1 · Panzer · 2010 [cited by applicant]
US 20110112855A1 · Chen et al. · 2011 [cited by applicant]
US 20110126207A1 · Wipfel et al. · 2011 [cited by applicant]
US 20120025989A1 · Cuddihy et al. · 2012 [cited by applicant]
US 20120075464A1 · Derenne et al. · 2012 [cited by applicant]
US 20120165618A1 · Algoo · 2012 [cited by examiner]
US 20120179067A1 · Wekell · 2012 [cited by applicant]
US 20120229634A1 · Laett et al. · 2012 [cited by applicant]
US 20130060167A1 · Dracup · 2013 [cited by applicant]
US 20130123667A1 · Komatireddy et al. · 2013 [cited by applicant]
US 20130127620A1 · Siebers et al. · 2013 [cited by applicant]
US 20130167025A1 · Patri et al. · 2013 [cited by applicant]
US 20130204545A1 · Solinsky · 2013 [cited by applicant]
US 20130212501A1 · Anderson et al. · 2013 [cited by applicant]
US 20130237395A1 · Hjelt et al. · 2013 [cited by applicant]
US 20130289449A1 · Stone et al. · 2013 [cited by applicant]
US 20130303860A1 · Bender et al. · 2013 [cited by applicant]
US 20140074454A1 · Brown et al. · 2014 [cited by applicant]
US 20140094136A1 · Huang · 2014 [cited by examiner]
US 20140112321A1 · Larson et al. · 2014 [cited by applicant]
US 20140112351A1 · Furukawa · 2014 [cited by examiner]
US 20140148733A1 · Stone et al. · 2014 [cited by applicant]
US 20140214441A1 · Young et al. · 2014 [cited by applicant]
US 20140232600A1 · Larose et al. · 2014 [cited by applicant]
US 20140243686A1 · Kimmel · 2014 [cited by applicant]
US 20140278605A1 · Borucki et al. · 2014 [cited by applicant]
US 20140317502A1 · Brown et al. · 2014 [cited by applicant]
US 20140337048A1 · Brown et al. · 2014 [cited by applicant]
US 20140343460A1 · Evans, III et al. · 2014 [cited by applicant]
US 20150005674A1 · Schindler · 2015 [cited by applicant]
US 20150109442A1 · Derenne et al. · 2015 [cited by applicant]
US 20150359467A1 · Tran · 2015 [cited by applicant]
US 20160037057A1 · Westin et al. · 2016 [cited by applicant]
US 20160125620A1 · Heinrich et al. · 2016 [cited by applicant]
US 20160154977A1 · Jagadish et al. · 2016 [cited by applicant]
US 20160156696A1 · Liddicott · 2016 [cited by examiner]
US 20160216770A1 · Jang et al. · 2016 [cited by applicant]
US 20160267699A1 · Borke et al. · 2016 [cited by applicant]
US 20170055917A1 · Stone et al. · 2017 [cited by applicant]
US 20170147154A1 · Steiner et al. · 2017 [cited by applicant]
US 20170189751A1 · Knickerbocker et al. · 2017 [cited by applicant]
US 20170192950A1 · Gaither et al. · 2017 [cited by applicant]
US 20170197115A1 · Cook et al. · 2017 [cited by applicant]
US 20170223176A1 · Anzures et al. · 2017 [cited by applicant]
US 20170251985A1 · Newton · 2017 [cited by applicant]
US 20170336933A1 · Hassel · 2017 [cited by applicant]
US 20170337274A1 · Ly et al. · 2017 [cited by applicant]
US 20180005448A1 · Choukroun et al. · 2018 [cited by applicant]
US 20180096504A1 · Valdivia et al. · 2018 [cited by applicant]
US 20180189756A1 · Purves et al. · 2018 [cited by applicant]
US 20180330810A1 · Gamarnik et al. · 2018 [cited by applicant]
US 20180360349A9 · Dohrmann et al. · 2018 [cited by applicant]
US 20180365383A1 · Bates · 2018 [cited by applicant]
US 20180365759A1 · Balzer et al. · 2018 [cited by applicant]
US 20190019573A1 · Lake et al. · 2019 [cited by applicant]
US 20190019582A1 · Wallis et al. · 2019 [cited by applicant]
US 20190116212A1 · Spinella-Mamo · 2019 [cited by applicant]
US 20190156575A1 · Korhonen · 2019 [cited by applicant]
US 20190176043A1 · Gosine et al. · 2019 [cited by applicant]
US 20190220727A1 · Dohrmann et al. · 2019 [cited by applicant]
US 20190259475A1 · Dohrmann et al. · 2019 [cited by applicant]
US 20200043594A1 · Miller et al. · 2020 [cited by applicant]
US 20200066391A1 · Sachdeva · 2020 [cited by applicant]
US 20200129107A1 · Sharma et al. · 2020 [cited by applicant]
US 20200236090A1 · De Beer et al. · 2020 [cited by applicant]
US 20210007631A1 · Dohrmann et al. · 2021 [cited by applicant]
US 20210016150A1 · Jeong et al. · 2021 [cited by applicant]
US 20210052230A1 · Roh · 2021 [cited by applicant]
US 20210110894A1 · Shriberg et al. · 2021 [cited by applicant]
US 20210134456A1 · Posnack et al. · 2021 [cited by applicant]
US 20210375426A1 · Gobezie et al. · 2021 [cited by applicant]
US 20220031199A1 · Hao et al. · 2022 [cited by applicant]
US 20220157427A1 · Keeley et al. · 2022 [cited by applicant]
US 20220199252A1 · Dohrmann et al. · 2022 [cited by applicant]
US 20220319696A1 · Dohrmann et al. · 2022 [cited by applicant]
US 20220319714A1 · Dohrmann et al. · 2022 [cited by applicant]
US 20220359091A1 · Dohrmann et al. · 2022 [cited by applicant]
US 20230108601A1 · Coelho Alves et al. · 2023 [cited by applicant]
CA 2949449A1 · 2015 [cited by applicant]
CN 104361321A · 2015 [cited by applicant]
CN 106056035A · 2016 [cited by applicant]
CN 106940692A · 2017 [cited by applicant]
CN 107411515A · 2017 [cited by applicant]
EP 3703009A1 · 2020 [cited by applicant]
JP 2000232963A · 2000 [cited by applicant]
JP 2002304362A · 2002 [cited by applicant]
JP 2005228305A · 2005 [cited by applicant]
JP 2008062071A · 2008 [cited by applicant]
JP 2008123318A · 2008 [cited by applicant]
JP 2008229266A · 2008 [cited by applicant]
JP 2016525383A · 2016 [cited by applicant]
JP 2017187914A · 2017 [cited by applicant]
KR 20170069501A · 2017 [cited by applicant]
WO WO2000005639A2 · 2000 [cited by applicant]
WO WO2014210344A1 · 2014 [cited by applicant]
Unity (game engine); Wikipedia website (Year: 2024). [cited by examiner]
Leber, Jessica, “The Avatar Will See You Now”, MIT Technology Review, Sep. 17, 2013, 4 pages. [cited by applicant]
Marston et al., “The design of a purpose-built exergame for fall prediction and prevention for older people”, European Review of Aging and Physical Activity 12:13, <URL:https://eurapa.biomedcentral.com/track/pdf/10.1186… [cited by applicant]
Ejupi et al., “Kinect-Based Five-Times-Sit-to-Stand Test for Clinical and In-Home Assessment of Fall Risk in Older People”, Gerontology (vol. 62), (2015-05-28), <URL:https://www.karger.com/Article/PDF/381804>, May 28, 2… [cited by applicant]
Festl et al., “iStoppFalls: A Tutorial Concept and prototype Contents”, <URL:https://hcisiegen.de/wp-uploads/2014/05/isCtutorialdoku.pdf>, Mar. 30, 2013, 36 pages. [cited by applicant]
Dubois et al., “A Gait Analysis Method Based on a Depth Camera for Fall Prevention,” Proc. of the 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), Aug. 30, 2014, pp. 4… [cited by applicant]
Marston et al., “The design of a purpose-built exergame for fall prediction and prevention for older people,” European Review of Aging and Physical Activity, Dec. 8, 2015, vol. 12, pp. 1-12. [cited by applicant]
Wasenmuller et al., “Comparison of Kinect V1 and V2 Depth Images in Terms of Accuracy and Precision”, Computer Vision—ACCV 2016 Workshops (Taipei, Taiwan, Nov. 20-24, 2016), Revised Selected Papers, Part II, Mar. 16, 20… [cited by applicant]
Stone et al., “Evaluation of an Inexpesive Depth Camera for In-Home Gait Assessment,” Journal of Ambient Intelligence and Smart Environments Jan. 2011 3(4); pp. 349-361. [cited by applicant]
Similan et al., Gait analysis and estimation of changes in fall risk factors,2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy, 2015, doi: 10.1109/EMB… [cited by applicant]