IP Library Granted Patent US 12,242,264
Granted Patent B2
US 12,242,264 · App. 17/978,035 · Granted Mar 4, 2025

Using neural network to optimize operational parameter of vehicle while achieving favorable emotional state of rider

Inventor: Charles Howard Cella (Pembroke, MA)
Assignee: Strong Force TP Portfolio 2022, LLC
G05D1/0022B60W40/08G01C21/3438G01C21/3461G01C21/3469G01C21/3617G05B13/027G05D1/0088G05D1/0212G05D1/0287G05D1/224G05D1/225G05D1/226G05D1/227G05D1/228G05D1/229G05D1/24G05D1/646G05D1/69G05D1/692G05D1/81G06F40/40G06N3/0418G06N3/045G06N3/08G06N3/086G06N20/00G06Q30/0208G06Q50/188G06Q50/40G06V10/764G06V10/82G06V20/56G06V20/59G06V20/597G06V20/64G07C5/006G07C5/008G07C5/02G07C5/08G07C5/0808G07C5/0816G07C5/0866G07C5/0891G10L15/16G10L25/63B60W2040/0881G06N3/02G06Q30/0281G06Q50/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,242,264
App. No.
17/978,035
Filed
Oct 31, 2022
Granted
Mar 4, 2025
Kind
B2
Art Unit
3628
USPC
705/80
Abstract

A system for transportation includes a vehicle, a rider occupying the vehicle, and a hybrid neural network. The hybrid neural network includes a first neural network to process a sensor input corresponding to the rider to determine an emotional state of the rider, and a second neural network to optimize at least one operating parameter of the vehicle to improve the emotional state of the rider.

Claims (30)

1. A system for transportation, comprising:

a vehicle interface to interface with a vehicle occupied by a rider; and

a hybrid neural network, including:

a first neural network to process a sensor input corresponding to the rider to determine an emotional state of the rider;

a second neural network to classify social media data sourced from a plurality of social media sources and as being indicative of an effect on the system for transportation, wherein the social media data indicates at least one of an interest of the rider or a transportation objective of the rider, and

a third neural network to analyze the classified social media data and optimize at least one operating parameter of the vehicle to improve the emotional state of the rider, wherein the analysis of the classified social media data by the third neural network includes determining a value for the at least one operating parameter that is likely to improve the emotional state of the rider based on at least one of a model or a rule, wherein the at least one of the model or the rule includes a weighting for the at least one operating parameter, and wherein the optimization of the at least one operating parameter includes using the vehicle interface for at least one of:

adjustment of a routing plan of the vehicle to include passing a point of interest to the rider, or

selecting entertainment content from entertainment options including at least one of: movies, games, or advertising content,

wherein the third neural network is configured to optimize the at least one operating parameter of the vehicle in part by varying the at least one operating parameter to vary a cortisol level of the rider based on a time of day.

2. The system for transportation of claim 1 wherein the at least one operating parameter of the vehicle that is optimized by the third neural network affects: a speed of the vehicle, an acceleration of the vehicle, a deceleration of the vehicle, and a proximity to other vehicles along a route.

3. The system for transportation of claim 1 wherein a sensor that produces the sensor input corresponding to the rider is to gather physiological sensed data indicative of a hormone level of the rider, wherein the physiological sensed data includes galvanic skin response that is indicative of the cortisol level of the rider.

4. The system for transportation of claim 3 wherein the first neural network is to determine the emotional state of the rider through analysis of the physiological sensed data.

5. The system for transportation of claim 3 wherein the sensor is deployed in the vehicle.

6. The system for transportation of claim 3 wherein the sensor is a wearable sensor.

7. The system for transportation of claim 1 wherein the third neural network is to optimize the at least one operating parameter of the vehicle based on a correlation between a vehicle operating state and the emotional state of the rider.

8. The system for transportation of claim 1 wherein the emotional state of the rider that is improved includes at least one aspect of rider satisfaction.

9. The system for transportation of claim 8 wherein the at least one aspect of rider satisfaction is one of; an operating state of the vehicle, an in-vehicle state, a rider state, a routing state, or a user experience state.

10. The system for transportation of claim 1 wherein at least one of the neural networks in the hybrid neural network is a convolutional neural network.

11. The system for transportation of claim 1 wherein the social media data includes at least one of:

social media posts, social media feeds, like or dislike activity detected in the social media, indications of relationships, user behavior, discussion threads, or chats.

12. The system for transportation of claim 1 wherein the social media data includes at least one of: presence of a rare phenomenon at a location, entertainment choices, risk-related conditions, or modes of transportation used with an event.

13. A method, comprising:

processing, using a first neural network of a hybrid neural network, a sensor input corresponding to a rider in a vehicle to determine an emotional state of the rider;

classifying, using a second neural network of the hybrid neural network, social media data sourced from a plurality of social media sources and as being indicative of an effect on a system for transportation that includes the vehicle having the rider therein and the hybrid neural network, wherein the social media data indicates at least one of an interest of the rider or a transportation objective of the rider, and

analyzing, using a third neural network of the hybrid neural network, the classified social media data and optimizing at least one operating parameter of the vehicle to improve the emotional state of the rider, wherein analyzing the classified social media data includes determining a value for the at least one operating parameter that is likely to improve the emotional state of the rider based on at least one of a model or a rule, wherein the at least one of the model or the rule includes a weighting for the at least one operating parameter, and wherein the optimizing of the at least one operating parameter includes at least one of:

adjusting a routing plan of the vehicle to include passing a point of interest to the rider, or

selecting entertainment content from entertainment options, wherein the entertainment options include at least one of movies, games, or advertising content,

and wherein the third neural network is used to optimize the at least one operating parameter of the vehicle in part by varying the at least one operating parameter to vary a cortisol level of the rider based on a time of day.

14. The method of claim 13 wherein the at least one operating parameter of the vehicle that is optimized by the third neural network affects: a speed of the vehicle, an acceleration of the vehicle, a deceleration of the vehicle, and a proximity to other vehicles along a route.

15. The method of claim 13 wherein a sensor that produces the sensor input corresponding to the rider is to gather physiological sensed data indicative of a hormone level of the rider, wherein the physiological sensed data includes galvanic skin response indicative of the cortisol level of the rider.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 2, 2022
From: CELLA, CHARLES HOWARD
To: STRONG FORCE TP PORTFOLIO 2022, LLC
Reel/Frame 061960/0760 →
Continuity (5)
Continuation 17113020 · Dec 5, 2020
Continuation 16694804 · Nov 25, 2019
Continuation PCTUS2019053857 · Sep 30, 2019
Provisional Application 62739335 · Sep 30, 2018
Related Publication 20230053340A1 · Feb 23, 2023
References Cited (154)
US 5400734A · Doyon · 1995 [cited by applicant]
US 6058352A · Lu et al. · 2000 [cited by applicant]
US 6188945B1 · Graf et al. · 2001 [cited by applicant]
US 9082239B2 · Ricci · 2015 [cited by applicant]
US 9451396B2 · Takatsuji · 2016 [cited by applicant]
US 9744363B2 · Grill, Jr. et al. · 2017 [cited by applicant]
US 9849044B1 · Groden et al. · 2017 [cited by applicant]
US 9868394B1 · Fields et al. · 2018 [cited by applicant]
US 10117109B2 · Madaiah et al. · 2018 [cited by applicant]
US 10122692B2 · MacCarthaigh · 2018 [cited by applicant]
US 10126743B2 · Fukumoto · 2018 [cited by applicant]
US 10289618B2 · Fink et al. · 2019 [cited by applicant]
US 10308258B2 · Fung et al. · 2019 [cited by applicant]
US 10346415B1 · Smith et al. · 2019 [cited by applicant]
US 10349223B1 · Yoo et al. · 2019 [cited by applicant]
US 10372130B1 · Kaushansky et al. · 2019 [cited by applicant]
US 10373522B2 · Byron et al. · 2019 [cited by applicant]
US 10387205B2 · Boutnaru · 2019 [cited by applicant]
US 10387417B1 · Hansen et al. · 2019 [cited by applicant]
US 10387429B2 · Smith-Mickelson et al. · 2019 [cited by applicant]
US 10423934B1 · Zanghi et al. · 2019 [cited by applicant]
US 10678846B2 · Gordo Soldevila et al. · 2020 [cited by applicant]
US 10942520B1 · Copeland et al. · 2021 [cited by applicant]
US 11042938B1 · Robare · 2021 [cited by applicant]
US 11334951B1 · Kolls et al. · 2022 [cited by applicant]
US 20050143845A1 · Kaji · 2005 [cited by applicant]
US 20060011399A1 · Brockway et al. · 2006 [cited by applicant]
US 20080195432A1 · Fell et al. · 2008 [cited by applicant]
US 20080219013A1 · Budinger et al. · 2008 [cited by applicant]
US 20080269958A1 · Filev et al. · 2008 [cited by applicant]
US 20080281516A1 · Cummings · 2008 [cited by applicant]
US 20080319645A1 · Kumagai et al. · 2008 [cited by applicant]
US 20100171441A1 · Schlangen et al. · 2010 [cited by applicant]
US 20100171767A1 · Waeller et al. · 2010 [cited by applicant]
US 20110213511A1 · Msconti et al. · 2011 [cited by applicant]
US 20110218835A1 · Boss et al. · 2011 [cited by applicant]
US 20130056312A1 · Finschi · 2013 [cited by applicant]
US 20130090802A1 · Curtis et al. · 2013 [cited by applicant]
US 20130110296A1 · Khoo et al. · 2013 [cited by applicant]
US 20130121591A1 · Hill · 2013 [cited by applicant]
US 20140058669A1 · Singh · 2014 [cited by applicant]
US 20140139655A1 · Mimar · 2014 [cited by applicant]
US 20140156341A1 · Kruk · 2014 [cited by applicant]
US 20140189002A1 · Orioli et al. · 2014 [cited by applicant]
US 20140195125A1 · Siegel et al. · 2014 [cited by applicant]
US 20140229255A1 · Scofield et al. · 2014 [cited by applicant]
US 20140229568A1 · Raffa et al. · 2014 [cited by applicant]
US 20140278031A1 · Scofield et al. · 2014 [cited by applicant]
US 20150025731A1 · Uehara · 2015 [cited by applicant]
US 20150204684A1 · Rostamian et al. · 2015 [cited by applicant]
US 20150206000A1 · El Kaliouby et al. · 2015 [cited by applicant]
US 20150210262A1 · Mitchell et al. · 2015 [cited by applicant]
US 20150314681A1 · Riley, Sr. et al. · 2015 [cited by applicant]
US 20150344036A1 · Kristinsson et al. · 2015 [cited by applicant]
US 20150348112A1 · Ramanujam · 2015 [cited by applicant]
US 20160019565A1 · Doganata et al. · 2016 [cited by applicant]
US 20160026936A1 · Richardson et al. · 2016 [cited by applicant]
US 20160104486A1 · Penilla et al. · 2016 [cited by applicant]
US 20160161275A1 · Waltermann et al. · 2016 [cited by applicant]
US 20160320198A1 · Liu et al. · 2016 [cited by applicant]
US 20160367157A1 · Blake et al. · 2016 [cited by applicant]
US 20170056764A1 · Shukla · 2017 [cited by applicant]
US 20170109944A1 · Barajas Gonzalez et al. · 2017 [cited by applicant]
US 20170163591A1 · Jimenez Pazmino et al. · 2017 [cited by applicant]
US 20170200449A1 · Penilla et al. · 2017 [cited by applicant]
US 20170242433A1 · Ochiai et al. · 2017 [cited by applicant]
US 20170262790A1 · Khasis · 2017 [cited by applicant]
US 20170299399A1 · Yamaguchi et al. · 2017 [cited by applicant]
US 20170344006A1 · Kasami et al. · 2017 [cited by applicant]
US 20170351990A1 · Hecht et al. · 2017 [cited by applicant]
US 20170352267A1 · Tzirkel-Hancock et al. · 2017 [cited by applicant]
US 20170372219A1 · Crawford et al. · 2017 [cited by applicant]
US 20180004211A1 · Grimm et al. · 2018 [cited by applicant]
US 20180005116A1 · Pickover et al. · 2018 [cited by applicant]
US 20180014182A1 · Jaegal et al. · 2018 [cited by applicant]
US 20180040163A1 · Donnelly et al. · 2018 [cited by applicant]
US 20180089605A1 · Poornachandran et al. · 2018 [cited by applicant]
US 20180115598A1 · Shariat et al. · 2018 [cited by applicant]
US 20180121847A1 · Zhao et al. · 2018 [cited by applicant]
US 20180126967A1 · Sanders · 2018 [cited by applicant]
US 20180134116A1 · Chen et al. · 2018 [cited by applicant]
US 20180137593A1 · Djuric · 2018 [cited by examiner]
US 20180174457A1 · Taylor · 2018 [cited by applicant]
US 20180209801A1 · Stentz et al. · 2018 [cited by applicant]
US 20180211337A1 · Ghaddar et al. · 2018 [cited by applicant]
US 20180225551A1 · Lin et al. · 2018 [cited by applicant]
US 20180229674A1 · Heinrich et al. · 2018 [cited by applicant]
US 20180232508A1 · Kursun · 2018 [cited by applicant]
US 20180252540A1 · Lim · 2018 [cited by applicant]
US 20180267541A1 · Goldberg et al. · 2018 [cited by applicant]
US 20180276351A1 · Patton et al. · 2018 [cited by applicant]
US 20180299276A1 · Sayer · 2018 [cited by applicant]
US 20180314250A1 · Lewis et al. · 2018 [cited by applicant]
US 20180322112A1 · Bellegarda et al. · 2018 [cited by applicant]
US 20180329428A1 · Nagy et al. · 2018 [cited by applicant]
US 20180330178A1 · El Kaliouby et al. · 2018 [cited by applicant]
US 20190011931A1 · Selvam et al. · 2019 [cited by applicant]
US 20190033085A1 · Ogale et al. · 2019 [cited by applicant]
US 20190034829A1 · Guim Bernat et al. · 2019 [cited by applicant]
US 20190038204A1 · Beck et al. · 2019 [cited by applicant]
US 20190043364A1 · Jumpertz · 2019 [cited by applicant]
US 20190047584A1 · Donnelly · 2019 [cited by applicant]
US 20190049267A1 · Huang · 2019 [cited by applicant]
US 20190050774A1 · Divine et al. · 2019 [cited by applicant]
US 20190065903A1 · Wang et al. · 2019 [cited by applicant]
US 20190113362A1 · Kline et al. · 2019 [cited by applicant]
US 20190120654A1 · Todasco et al. · 2019 [cited by applicant]
US 20190122544A1 · Schlesinger et al. · 2019 [cited by applicant]
US 20190139165A1 · Valecha et al. · 2019 [cited by applicant]
US 20190143997A1 · Crimando et al. · 2019 [cited by applicant]
US 20190185009A1 · Werner et al. · 2019 [cited by applicant]
US 20190225232A1 · Blau · 2019 [cited by applicant]
US 20190236959A1 · Belapurkar et al. · 2019 [cited by applicant]
US 20190248361A1 · Farrell et al. · 2019 [cited by applicant]
US 20190286162A1 · Fischer et al. · 2019 [cited by applicant]
US 20190294173A1 · Szubbocsev · 2019 [cited by applicant]
US 20190311416A1 · Pandit et al. · 2019 [cited by applicant]
US 20190311614A1 · Yang et al. · 2019 [cited by applicant]
US 20190315342A1 · Bertollini et al. · 2019 [cited by applicant]
US 20190325376A1 · Khasis · 2019 [cited by applicant]
US 20200073407A1 · He et al. · 2020 [cited by applicant]
US 20200104962A1 · Aich et al. · 2020 [cited by applicant]
US 20200107176A1 · Zavesky et al. · 2020 [cited by applicant]
US 20200284594A1 · Wang et al. · 2020 [cited by applicant]
US 20200356608A1 · Ogawa et al. · 2020 [cited by applicant]
US 20210035074A1 · Alieiev et al. · 2021 [cited by applicant]
US 20220091673A1 · Fujimaki et al. · 2022 [cited by applicant]
CN 103236163A · 2013 [cited by applicant]
EP 2942012A1 · 2015 [cited by applicant]
JP 2017213984A · 2017 [cited by applicant]
KR 20150050134A · 2015 [cited by applicant]
WO 2009061687A1 · 2009 [cited by applicant]
WO 2016108873A1 · 2016 [cited by applicant]
WO 2018217640A1 · 2018 [cited by applicant]
Collins, et al., A Novel Transit Rider Satisfaction Metric: Rider Sentiments Measured from Online Social Media Data, Journal of Public Transportation, vol. 16, No. 2, Apr. 2013, pp. 21-45 (Year: 2013). [cited by examiner]
Evans, et al., Rail commuting duration and passenger stress, Health Psychology, vol. 25, No. 3, 2006, pp. 408-412. (Year: 2006). [cited by examiner]
Novaco, et al., Commuting and well-being, Technology and Well-Being, vol. 3, 2009, 174-4. (Year: 2009). [cited by examiner]
Villarejo, et al., “A Stress Sensor Based on Galvanic Skin Response (GSR) Controlled by ZigBee Sensors”, Sensors, 12, 6075-6101, 2012. [cited by applicant]
Nguyen, Thi, “ETA Phone Home: How Uber Engineers an Efficient Route”, [online] Uber, published on Nov. 3, 2015, available at https://eng.uber.com/engineering-routing-engine/, 2015. [cited by applicant]
Nguyen, Hoang, et al., “TrafficWatch: Real-Time Traffic Incident Detection and Monitoring Using Social Media”, PAKDD 2016: Advances in Knowledge Discovery and Data Mining pp. 540-551, Apr. 12, 2016. [cited by applicant]
Bloom, Cara, et al., “Self-Driving Cars and Data Collection: Privacy Perceptions of Networked Autonomous Vehicles”, Proceedings of the Thirteenth Symposium on Usable Privacy and Security, Jul. 12-14, 2017, 21 pages, 201… [cited by applicant]
McGrath, Stevan, “Artificial Intelligence and Social Media: Will AI Transform Social Media Marketing?”, rocksdigital.com, 18 pages, Nov. 28, 2017. [cited by applicant]
Reutzel, “How Blockchain Tech Could Move Self-Driving Cars into the Fast Lane”, available at https://www.coindesk.com/markets/2017/05/25/how-blockchain-tech-could-move-self-driving-cars-into-the-fast-lane/, May 25, 2017. [cited by applicant]
Author Unknown, “The Difference Between Robotic Process Automation and Artificial Intelligence”, CFB Bots, retrieved from https://medium.com/@cfb_bots, 7 pages, Apr. 20, 2018. [cited by applicant]
Gallego, “Stress Level Assessment with Non-Intrusive Sensors, Doctoral Thesis”, Universidad Carlos III De Madrid, accessed at http://www.tsc.uc3m.es/˜fhernando/ research/ ThesisFHG.pdf on Jun. 21, 2021, Apr. 2018. [cited by applicant]
Gonzalez, Iris, “FunnelAI's Artificial Intelligence Finds Real-time Customer Leads in Social Media”, Startups San Antonio, https://www.startupssanantonio.com/funnelai-s-artificial-intelligence-finds-real-time-customer-l… [cited by applicant]
Yang, Carl, et al., “Did You Enjoy the Ride: Understanding Passenger Experience via Heterogeneous Network Embedding”, 2018 IEEE 34th International Conference on Data Engineering, pp. 1392-1403, 2018. [cited by applicant]
Sekar, M., et al., “Carbon fiber based electrochemical sensor for sweat cortisol measurement”, Scientific Reports, 9:403-403, 2019. [cited by applicant]
Tomczak et al., “Drying kinetics simulation by means of artificial neural networks”, in Handbook of Conveying and Handling of Particulate Solids, A. Levy and H. Kalman (Eds.), Elsevier Science B.V., pp. 569-580, 2001. [cited by applicant]
Wener et al., “Commuting Stress: Psychophysiological Effects of a Trip and Spillover into the Workplace”, Transportation Research Record: Journal of the Transportation Research Board, No. 1924, Transportation Research B… [cited by applicant]
Rouse, “Multilayer Perceptron”, Technopedia, Mar. 30, 2017, https://www.techopedia.com/definition/20879/multilayer-perceptron-mlp, 2017. [cited by applicant]
Author Unknown, “What is Multilayer Perceptron (MLP) Neural Networks?”, Shiska Online, https://www.shiksha.com/online-courses/articles/understanding-multilayer-perceptron-mlp-neural-networks/ [Updated Feb. 9, 2023], 202… [cited by applicant]
Author Unknown, “1.17.1. Multi-layer Perceptron in 1.17. Neural network models (supervised)”, scikit learn, https://scikit-learn.org/stable/modules/neural_networks_supervised.html, 2023. [cited by applicant]
Hu et al., “Behavior Insights for an Incentive-Based Active Demand Management Platform”, International Journal of Transportation Science and Technology, vol. 4, No. 2, Jun. 2015, pp. 119-134, 2015. [cited by applicant]