IP Library Granted Patent US 12,216,466
Granted Patent B2
US 12,216,466 · App. 18/396,429 · Granted Feb 4, 2025

Method of maintaining a favorable emotional state of a rider of a vehicle by a neural network to classify emotional state indicative wearable sensor data

Inventor: Charles Howard Cella (Pembroke, MA)
Assignee: Strong Force TP Portfolio 2022, LLC
G05D1/0022B60W40/08G01C21/3438G01C21/3461G01C21/3469G01C21/3617G05B13/027G05D1/0088G05D1/0212G05D1/0287G05D1/224G05D1/225G05D1/226G05D1/227G05D1/228G05D1/229G05D1/24G05D1/646G05D1/69G05D1/692G05D1/81G06F40/40G06N3/0418G06N3/045G06N3/08G06N3/086G06N20/00G06Q30/0208G06Q50/188G06Q50/40G06V10/764G06V10/82G06V20/56G06V20/59G06V20/597G06V20/64G07C5/006G07C5/008G07C5/02G07C5/08G07C5/0808G07C5/0816G07C5/0866G07C5/0891G10L15/16G10L25/63B60W2040/0881G06N3/02G06Q30/0281G06Q50/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,216,466
App. No.
18/396,429
Filed
Dec 26, 2023
Granted
Feb 4, 2025
Kind
B2
Art Unit
3694
USPC
705/14.11
Abstract

A method of maintaining a favorable emotional state of a rider of a vehicle. The method includes: receiving emotional state indicative wearable sensor data of the rider in the vehicle; classifying, at a rider emotional state determining neural network, the emotional state indicative wearable sensor data and generating output data indicative of a classification of the rider emotional state as one of a favorable emotional state of the rider or an unfavorable emotional state of the rider; converting the output data from the rider emotional state determining neural network into a vehicle operational state datum; and adjusting a vehicle operating parameter in response to the vehicle operational state datum.

Claims (33)

1. A method of maintaining a favorable emotional state of a rider in a vehicle, the method comprising:

receiving emotional state indicative wearable sensor data of the rider;

classifying, at a rider emotional state determining neural network, the emotional state indicative wearable sensor data and generating output data indicative of a classification of an emotional state as one of: the favorable emotional state of the rider; or an unfavorable emotional state of the rider;

converting the output data from the rider emotional state determining neural network into a vehicle operational state datum; and

adjusting a vehicle operating parameter in response to the vehicle operational state datum.

2. The method of claim 1 wherein the output data is indicative of at least one of the emotional state of the rider is changing, the emotional state of the rider is stable, a rate of change of the emotional state of the rider, a direction of change of the emotional state of the rider, or a polarity of a change of the emotional state of the rider.

3. The method of claim 1 wherein the output data is indicative of one of: the emotional state of the rider is changing to an unfavorable state; or the emotional state of the rider is changing to a favorable state.

4. The method of claim 1 wherein the vehicle operating parameter includes at least one of: a route of the vehicle, an in-vehicle audio content, a speed of the vehicle, an acceleration of the vehicle, a deceleration of the vehicle, a proximity to objects along the route, or a proximity to other vehicles along the route.

5. The method of claim 1 wherein the receiving emotional state indicative wearable sensor data comprises:

receiving the emotional state indicative wearable sensor data from at least one of: a watch, a ring, a wrist band, an arm band, an ankle band, a torso band, a skin patch, a head-worn device, eyeglasses, foot wear, a glove, an in-ear device, clothing, headphones, a belt, a finger ring, a thumb ring, a toe ring, or a necklace.

6. The method of claim 1 wherein the receiving emotional state indicative wearable sensor data comprises:

detecting, at the rider emotional state determining neural network, the emotional state of the rider through expert system-based processing of the emotional state indicative wearable sensor data of a plurality of wearable physiological condition sensors worn by the rider in the vehicle.

7. The method of claim 6 wherein the plurality of wearable physiological condition sensors comprises at least two of: a watch, a ring, a wrist band, an arm band, an ankle band, a torso band, a skin patch, a head-worn device, eyeglasses, foot wear, a glove, an in-ear device, clothing, headphones, a belt, a finger ring, a thumb ring, a toe ring, or a necklace.

8. The method of claim 1 wherein the vehicle is a self-driving vehicle.

9. The method of claim 1 wherein classifying the emotional state indicative wearable sensor data includes classifying the emotional state of the rider as the favorable emotional state, and wherein the method further comprises

optimizing the vehicle operating parameter to maintain the favorable emotional state.

10. The method of claim 1 wherein classifying the emotional state indicative wearable sensor data includes using at least one of: a training set of emotional state indicators of a set of riders or trainer-generated rider emotional state indicators.

11. A data processing system for maintaining a favorable emotional state of a rider of a vehicle, the data processing system comprising:

a processor;

a wearable sensor worn by the rider configured to communicate emotional state indicative wearable sensor data;

a neural network configured to receive the emotional state indicative wearable sensor data of the rider; and

a rider emotional state determining neural network configured to classify the emotional state indicative wearable sensor data and to generate output data indicative of a classification as one of: the favorable emotional state of the rider; or an unfavorable emotional state of the rider;

wherein the neural network is configured to convert the output data from the rider emotional state determining neural network into vehicle operational state data, and to adjust a vehicle operating parameter in response to the vehicle operational state data.

12. The data processing system of claim 11 wherein the output data is indicative of at least one of: an emotional state of the rider is changing, the emotional state of the rider is stable, a rate of change of the emotional state of the rider, a direction of change of the emotional state of the rider, or a polarity of a change of the emotional state of the rider.

13. The data processing system of claim 11 wherein the output data is indicative of one of: an emotional state of the rider is changing to an unfavorable state; or the emotional state of the rider is changing to a favorable state.

14. The data processing system of claim 11 wherein the vehicle operating parameter includes at least one of: a route of the vehicle, an in-vehicle audio content, a speed of the vehicle, an acceleration of the vehicle, a deceleration of the vehicle, a proximity to objects along the route, or a proximity to other vehicles along the route.

15. The data processing system of claim 11 wherein the emotional state indicative wearable sensor data comprises:

the emotional state indicative wearable sensor data from at least one of: a watch, a ring, a wrist band, an arm band, an ankle band, a torso band, a skin patch, a head-worn device, eyeglasses, foot wear, a glove, an in-ear device, clothing, headphones, a belt, a finger ring, a thumb ring, a toe ring, or a necklace.

16. The data processing system of claim 11 wherein the neural network detects an emotional state of the rider through expert system-based processing of the emotional state indicative wearable sensor data of a plurality of wearable physiological condition sensors worn by the rider.

17. The data processing system of claim 16 wherein the plurality of wearable physiological condition sensors comprise at least two of: a watch, a ring, a wrist band, an arm band, an ankle band, a torso band, a skin patch, a head-worn device, eyeglasses, foot wear, a glove, an in-ear device, clothing, headphones, a belt, a finger ring, a thumb ring, a toe ring, or a necklace.

18. The data processing system of claim 11 wherein the vehicle is a self-driving vehicle.

19. The data processing system of claim 11 wherein the rider emotional state determining neural network is further configured to optimize the vehicle operating parameter to maintain the favorable emotional state.

20. The data processing system of claim 11 wherein the rider emotional state determining neural network is configured to classify the emotional state indicative wearable sensor data by using at least one of: a training set of emotional state indicators of a set of riders; or trainer-generated rider emotional state indicators.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 24, 2024
From: CELLA, CHARLES HOWARD
To: STRONG FORCE TP PORTFOLIO 2022, LLC
Reel/Frame 066235/0543 →
Continuity (4)
Continuation 16803154 · Feb 27, 2020
Continuation PCTUS2019053857 · Sep 30, 2019
Provisional Application 62739335 · Sep 30, 2018
Related Publication 20240142975A1 · May 2, 2024
References Cited (154)
US 5400734A · Doyon · 1995 [cited by applicant]
US 6058352A · Lu et al. · 2000 [cited by applicant]
US 6188945B1 · Graf et al. · 2001 [cited by applicant]
US 9082239B2 · Ricci · 2015 [cited by applicant]
US 9451396B2 · Takatsjui · 2016 [cited by applicant]
US 9744363B2 · Grill, Jr. et al. · 2017 [cited by applicant]
US 9849044B1 · Groden et al. · 2017 [cited by applicant]
US 9868394B1 · Fields et al. · 2018 [cited by applicant]
US 10117109B2 · Madaiah et al. · 2018 [cited by applicant]
US 10122692B2 · Maccarthaigh · 2018 [cited by applicant]
US 10126743B2 · Fukumoto · 2018 [cited by applicant]
US 10289618B2 · Fink et al. · 2019 [cited by applicant]
US 10308258B2 · Fung et al. · 2019 [cited by applicant]
US 10346415B1 · Smith et al. · 2019 [cited by applicant]
US 10349223B1 · Yoo et al. · 2019 [cited by applicant]
US 10372130B1 · Kaushansky et al. · 2019 [cited by applicant]
US 10373522B2 · Byron et al. · 2019 [cited by applicant]
US 10387205B2 · Boutnaru · 2019 [cited by applicant]
US 10387417B1 · Hansen et al. · 2019 [cited by applicant]
US 10387429B2 · Smith-Mickelson et al. · 2019 [cited by applicant]
US 10423934B1 · Zanghi et al. · 2019 [cited by applicant]
US 10678846B2 · Gordo Soldevila et al. · 2020 [cited by applicant]
US 10942520B1 · Copeland et al. · 2021 [cited by applicant]
US 11042938B1 · Robare · 2021 [cited by applicant]
US 11334951B1 · Kolls et al. · 2022 [cited by applicant]
US 20050143845A1 · Kaji · 2005 [cited by applicant]
US 20060011399A1 · Brockway et al. · 2006 [cited by applicant]
US 20080195432A1 · Fell et al. · 2008 [cited by applicant]
US 20080219013A1 · Budinger et al. · 2008 [cited by applicant]
US 20080269958A1 · Filev et al. · 2008 [cited by applicant]
US 20080281516A1 · Cummings · 2008 [cited by applicant]
US 20080319645A1 · Kumagai et al. · 2008 [cited by applicant]
US 20100171441A1 · Schlangen et al. · 2010 [cited by applicant]
US 20100171767A1 · Waeller et al. · 2010 [cited by applicant]
US 20110213511A1 · Visconti et al. · 2011 [cited by applicant]
US 20110218835A1 · Boss et al. · 2011 [cited by applicant]
US 20130056312A1 · Finschi · 2013 [cited by applicant]
US 20130090802A1 · Curtis et al. · 2013 [cited by applicant]
US 20130110296A1 · Khoo et al. · 2013 [cited by applicant]
US 20130121591A1 · Hill · 2013 [cited by applicant]
US 20140058669A1 · Singh · 2014 [cited by applicant]
US 20140139655A1 · Mimar · 2014 [cited by applicant]
US 20140156341A1 · Kruk · 2014 [cited by applicant]
US 20140189002A1 · Orioli et al. · 2014 [cited by applicant]
US 20140195125A1 · Siegel et al. · 2014 [cited by applicant]
US 20140229255A1 · Scofield et al. · 2014 [cited by applicant]
US 20140229568A1 · Raffa et al. · 2014 [cited by applicant]
US 20140278031A1 · Scofield et al. · 2014 [cited by applicant]
US 20150025731A1 · Uehara · 2015 [cited by applicant]
US 20150204684A1 · Rostamian et al. · 2015 [cited by applicant]
US 20150206000A1 · El Kaliouby et al. · 2015 [cited by applicant]
US 20150210262A1 · Mitchell et al. · 2015 [cited by applicant]
US 20150314681A1 · Riley, Sr. et al. · 2015 [cited by applicant]
US 20150344036A1 · Kristinsson et al. · 2015 [cited by applicant]
US 20150348112A1 · Ramanujam · 2015 [cited by applicant]
US 20160019565A1 · Doganata et al. · 2016 [cited by applicant]
US 20160026936A1 · Richardson et al. · 2016 [cited by applicant]
US 20160104486A1 · Penilla et al. · 2016 [cited by applicant]
US 20160161275A1 · Waltermann et al. · 2016 [cited by applicant]
US 20160320198A1 · Liu et al. · 2016 [cited by applicant]
US 20160367157A1 · Blake et al. · 2016 [cited by applicant]
US 20170056764A1 · Shukla · 2017 [cited by applicant]
US 20170109944A1 · Barajas Gonzalez et al. · 2017 [cited by applicant]
US 20170163591A1 · Jimenez Pazmino et al. · 2017 [cited by applicant]
US 20170200449A1 · Penilla et al. · 2017 [cited by applicant]
US 20170242433A1 · Ochiai et al. · 2017 [cited by applicant]
US 20170262790A1 · Khasis · 2017 [cited by applicant]
US 20170299399A1 · Yamaguchi et al. · 2017 [cited by applicant]
US 20170344006A1 · Kasami et al. · 2017 [cited by applicant]
US 20170351990A1 · Hecht et al. · 2017 [cited by applicant]
US 20170352267A1 · Tzirkel-Hancock et al. · 2017 [cited by applicant]
US 20170372219A1 · Crawford et al. · 2017 [cited by applicant]
US 20180004211A1 · Grimm et al. · 2018 [cited by applicant]
US 20180005116A1 · Pickover · 2018 [cited by examiner]
US 20180014182A1 · Jaegal et al. · 2018 [cited by applicant]
US 20180040163A1 · Donnelly et al. · 2018 [cited by applicant]
US 20180089605A1 · Poornachandran et al. · 2018 [cited by applicant]
US 20180115598A1 · Shariat et al. · 2018 [cited by applicant]
US 20180121847A1 · Zhao et al. · 2018 [cited by applicant]
US 20180126967A1 · Sanders · 2018 [cited by applicant]
US 20180134116A1 · Chen et al. · 2018 [cited by applicant]
US 20180137593A1 · Djuric et al. · 2018 [cited by applicant]
US 20180174457A1 · Taylor · 2018 [cited by applicant]
US 20180209801A1 · Stentz et al. · 2018 [cited by applicant]
US 20180211337A1 · Ghaddar et al. · 2018 [cited by applicant]
US 20180225551A1 · Lin · 2018 [cited by examiner]
US 20180229674A1 · Heinrich et al. · 2018 [cited by applicant]
US 20180232508A1 · Kursun · 2018 [cited by applicant]
US 20180252540A1 · Lim · 2018 [cited by applicant]
US 20180267541A1 · Goldberg · 2018 [cited by applicant]
US 20180276351A1 · Patton et al. · 2018 [cited by applicant]
US 20180299276A1 · Sayer · 2018 [cited by applicant]
US 20180314250A1 · Lewis et al. · 2018 [cited by applicant]
US 20180322112A1 · Bellegarda et al. · 2018 [cited by applicant]
US 20180329428A1 · Nagy et al. · 2018 [cited by applicant]
US 20180330178A1 · El Kaliouby et al. · 2018 [cited by applicant]
US 20190011931A1 · Selvam et al. · 2019 [cited by applicant]
US 20190033085A1 · Ogale et al. · 2019 [cited by applicant]
US 20190034829A1 · Guim Bernat et al. · 2019 [cited by applicant]
US 20190038204A1 · Beck et al. · 2019 [cited by applicant]
US 20190043364A1 · Jumpertz · 2019 [cited by applicant]
US 20190047584A1 · Donnelly · 2019 [cited by applicant]
US 20190049267A1 · Huang · 2019 [cited by applicant]
US 20190050774A1 · Divine et al. · 2019 [cited by applicant]
US 20190065903A1 · Wang et al. · 2019 [cited by applicant]
US 20190113362A1 · Kline et al. · 2019 [cited by applicant]
US 20190120654A1 · Todasco et al. · 2019 [cited by applicant]
US 20190122544A1 · Schlesinger et al. · 2019 [cited by applicant]
US 20190139165A1 · Valecha et al. · 2019 [cited by applicant]
US 20190143997A1 · Crimando et al. · 2019 [cited by applicant]
US 20190185009A1 · Werner · 2019 [cited by examiner]
US 20190225232A1 · Blau · 2019 [cited by applicant]
US 20190236959A1 · Belapurkar et al. · 2019 [cited by applicant]
US 20190248361A1 · Farrell et al. · 2019 [cited by applicant]
US 20190286162A1 · Fischer · 2019 [cited by applicant]
US 20190294173A1 · Szubbocsev · 2019 [cited by applicant]
US 20190311416A1 · Pandit et al. · 2019 [cited by applicant]
US 20190311614A1 · Yang et al. · 2019 [cited by applicant]
US 20190315342A1 · Bertollini et al. · 2019 [cited by applicant]
US 20190325376A1 · Khasis · 2019 [cited by applicant]
US 20200073407A1 · He et al. · 2020 [cited by applicant]
US 20200104962A1 · Aich et al. · 2020 [cited by applicant]
US 20200107176A1 · Zavesky et al. · 2020 [cited by applicant]
US 20200284594A1 · Wang et al. · 2020 [cited by applicant]
US 20200356608A1 · Ogawa et al. · 2020 [cited by applicant]
US 20210035074A1 · Alieiev et al. · 2021 [cited by applicant]
US 20220091673A1 · Fujimaki et al. · 2022 [cited by applicant]
CN 103236163A · 2013 [cited by applicant]
EP 2942012A1 · 2015 [cited by applicant]
JP 2017213984A · 2017 [cited by applicant]
KR 20150050134A · 2015 [cited by applicant]
WO 2009061687A1 · 2009 [cited by applicant]
WO 2016108873A1 · 2016 [cited by applicant]
WO 2018217640A1 · 2018 [cited by applicant]
Tomczak et al., “Drying kinetics simulation by means of artificial neural networks”, in Handbook of Conveying and Handling of Particulate Solids, A. Levy and H. Kalman (Eds.), Elsevier Science B.V., pp. 569-580, 2001. [cited by applicant]
Wener et al., “Commuting Stress: Psychophysiological Effects of a Trip and Spillover into the Workplace”, Transportation Research Record: Journal of the Transportation Research Board, No. 1924, Transportation Research B… [cited by applicant]
Villarejo, et al., “A Stress Sensor Based on Galvanic Skin Response (GSR) Controlled by ZigBee Sensors”, Sensors, 12, 6075-6101, 2012. [cited by applicant]
Collins et al., “A Novel Transit: Rider Sentiments Measured from Online Social Media Data”, Journal of Public Transportation, vol. 16, No. 2, Apr. 2013, pp. 21-45, 2013. [cited by applicant]
Nguyen, Thi, “ETA Phone Home: How Uber Engineers an Efficient Route”, [online] Uber, published on Nov. 3, 2015, available at https://eng.uber.com/engineering-routing-engine/, 2015. [cited by applicant]
Nguyen, Hoang, et al., “TrafficWatch: Real-Time Traffic Incident Detection and Monitoring Using Social Media”, PAKDD 2016: Advances in Knowledge Discovery and Data Mining pp. 540-551 [First published online Apr. 12, 201… [cited by applicant]
Rouse, “Multilayer Perceptron”, Technopedia, Mar. 30, 2017, https://www.techopedia.com/definition/20879/multilayer-perceptron-mlp, 2017. [cited by applicant]
Reutzel, “How Blockchain Tech Could Move Self-Driving Cars into the Fast Lane”, available at https://www.coindesk.com/markets/2017/05/25/how-blockchain-tech-could-move-self-driving-cars-into-the-fast-lane/, May 25, 2017. [cited by applicant]
Bloom, Cara, et al., “Self-Driving Cars and Data Collection: Privacy Perceptions of Networked Autonomous Vehicles”, Proceedings of the Thirteenth Symposium on Usable Privacy and Security, Jul. 12-14, 2017, 21 pages, 201… [cited by applicant]
McGrath, Stevan, “Artificial Intelligence and Social Media: Will AI Transform Social Media Marketing?”, rocksdigital.com, 18 pages, Nov. 28, 2017. [cited by applicant]
Gonzalez, Iris, “FunnelAI's Artificial Intelligence Finds Real-time Customer Leads in Social Media”, Startups San Antonio, https://www.startupssanantonio.com/funnelai-s-artificial-intelligence-finds-real-time-customer-l… [cited by applicant]
Gallego, “Stress Level Assessment with Non-Intrusive Sensors, Doctoral Thesis”, Universidad Carlos III De Madrid, accessed at http://www.tsc.uc3m.es/˜fhernando/ research/ ThesisFHG.pdf on Jun. 21, 2021, Apr. 2018. [cited by applicant]
Author Unknown, “The Difference Between Robotic Process Automation and Artificial Intelligence”, CFB Bots, retrieved from https://medium.com/@cfb_bots, 7 pages, Apr. 20, 2018. [cited by applicant]
Yang, Carl, et al., “Did You Enjoy the Ride: Understanding Passenger Experience via Heterogeneous Network Embedding”, 2018 IEEE 34th International Conference on Data Engineering, pp. 1392-1403, 2018. [cited by applicant]
Sekar, M., et al., “Carbon fiber based electrochemical sensor for sweat cortisol measurement”, Scientific Reports, 9:403-403, 2019. [cited by applicant]
Author Unknown, “What is Multilayer Perceptron (MLP) Neural Networks?”, Shiska Online, https://www.shiksha.com/online-courses/articles/understanding-multilayer-perceptron-mlp-neural-networks/ [Updated Feb. 9, 2023], 202… [cited by applicant]
Author Unknown, “1.17.1. Multi-layer Perceptron in 1.17. Neural network models (supervised)”, scikit learn, https://scikit-learn.org/stable/modules/neural_networks_supervised.html, 2023. [cited by applicant]
Hu et al., “Behavior Insights for an Incentive-Based Active Demand Management Platform”, International Journal of Transportation Science and Technology, vol. 4, No. 2, Jun. 2015, pp. 119-134, 2015. [cited by applicant]
Evans et al., “Rail Commuting Duration and Passenger Stress”, Health Psychology, 2006, vol. 25, No. 3, p. 408-412, 2006. [cited by applicant]
Novaco et al., “Commuting and Well-Being”, chapter to appear in Technology and Well-Being, Yair Amichai-Hamburger (Ed.), Cambridge University Press, Dec. 22, 2009, 49 pages, 2009. [cited by applicant]