IP Library Granted Patent US 12,222,714
Granted Patent B2
US 12,222,714 · App. 18/395,424 · Granted Feb 11, 2025

Robotic process automation system trained to undertake actions with a vehicle based on user interactions with a user interface of the vehicle

Inventor: Charles Howard Cella (Pembroke, MA)
Assignee: Strong Force TP Portfolio 2022, LLC
G05D1/0022B60W40/08G01C21/3438G01C21/3461G01C21/3469G01C21/3617G05B13/027G05D1/0088G05D1/0212G05D1/0287G05D1/224G05D1/225G05D1/226G05D1/227G05D1/228G05D1/229G05D1/24G05D1/646G05D1/69G05D1/692G05D1/81G06F40/40G06N3/0418G06N3/045G06N3/08G06N3/086G06N20/00G06Q30/0208G06Q50/188G06Q50/40G06V10/764G06V10/82G06V20/56G06V20/59G06V20/597G06V20/64G07C5/006G07C5/008G07C5/02G07C5/08G07C5/0808G07C5/0816G07C5/0866G07C5/0891G10L15/16G10L25/63B60W2040/0881G06N3/02G06Q30/0281G06Q50/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,222,714
App. No.
18/395,424
Filed
Dec 22, 2023
Granted
Feb 11, 2025
Kind
B2
Art Unit
3668
USPC
701/23
Abstract

A method of operating a vehicle in a transportation system includes: receiving data collected for each of a set of users based on interactions of the set of users with a user interface of the vehicle including at least one of a vehicle data system, a vision system, and a connected system; performing robotic process automation on the interactions; learning changes in driving style using an artificial intelligence system based on the robotic process automation; and undertaking an action within the vehicle on behalf of the user based on the learned changes in driving style.

Claims (29)

1. A method of operating a vehicle in a transportation system, the method comprising:

receiving data collected for each of a set of users based on interactions of the set of users with a user interface of the vehicle including at least one of a vehicle data system, a vision system, and a connected system;

performing robotic process automation on the interactions to learn a driving style for at least one user of the set of users;

learning changes in the driving style using an artificial intelligence system based on the robotic process automation and based on feedback of the robotic process automation; and

optimizing a margin of safety and controlling the vehicle with the optimized margin of safety on behalf of the at least one user of the set of users based on the learned changes in the driving style, wherein the optimizing and the controlling affect at least one of: a route of the vehicle, a speed of the vehicle, an acceleration of the vehicle, a deceleration of the vehicle, a proximity to other vehicles along the route of the vehicle, a proximity to objects along the route of the vehicle, braking intensity, steering sensitivity, or route selection.

2. The method of claim 1 , wherein the data collected for each of the set of users includes at least one of: driving speed, braking patterns, steering patterns, or route preferences.

3. The method of claim 1 , wherein the artificial intelligence system is further trained using data from external sources including at least one of: traffic data, weather data, or road condition data.

4. The method of claim 1 , wherein the artificial intelligence system uses machine learning algorithms to learn the changes in the driving style.

5. The method of claim 1 , wherein the user interface of the vehicle includes at least one of: a touch screen display, voice recognition system, or a gesture recognition system.

6. The method of claim 1 , wherein the learned changes in the driving style are stored in at least one user profile associated with the at least one user of the set of users.

7. The method of claim 1 , wherein a robotic process automation system provides audio or visual feedback to the at least one user regarding the learned changes in the driving style.

8. The method of claim 1 , wherein an action undertaken within the vehicle during the optimizing and the controlling is based on real-time data collected during a current driving session.

9. The method of claim 1 , wherein the artificial intelligence system is further trained to predict future changes in the driving style based on historical data.

10. A method of robotic process automation to facilitate mimicking human operation of a vehicle, the method comprising:

tracking human interactions with a vehicle control-facilitating interface;

recording the tracked human interactions in a robotic process automation training data structure;

tracking vehicle operational state information of the vehicle, wherein the vehicle is controlled through the vehicle control-facilitating interface;

recording the vehicle operational state information in the robotic process automation training data structure;

training, through use of at least one neural network, an artificial intelligence system to operate the vehicle in a manner consistent with the human interactions based on the human interactions and the vehicle operational state information in the robotic process automation training data structure and based on feedback obtained from the robotic process automation training data structure,

wherein the training of the artificial intelligence system further involves training the artificial intelligence system to optimize a margin of safety and to control the vehicle on behalf of a user of the vehicle, wherein the optimizing and the controlling affect at least one of: a route of the vehicle, a speed of the vehicle, an acceleration of the vehicle, a deceleration of the vehicle, a proximity to other vehicles along the route of the vehicle, a proximity to objects along the route of the vehicle, braking intensity, steering sensitivity, or route selection; and

controlling, via the trained artificial intelligence system, the vehicle with the optimized margin of safety.

11. The method of claim 10 , wherein the human interactions with the vehicle control-facilitating interface include at least one of: steering inputs, braking inputs, acceleration inputs, or route selection inputs.

12. The method of claim 10 , wherein the vehicle operational state information includes at least one of: the speed of the vehicle, a location of the vehicle, a direction of the vehicle, a fuel level of the vehicle, or a battery level of the vehicle.

13. The method of claim 10 , wherein the at least one neural network is a deep learning neural network.

14. The method of claim 10 , wherein the vehicle control-facilitating interface includes at least one of: a touch screen display, a steering wheel, a brake pedal, an accelerator pedal, or a navigation system.

15. The method of claim 10 , wherein the robotic process automation training data structure is stored in a non-transitory computer-readable medium.

16. The method of claim 10 , wherein the training of the artificial intelligence system further includes reinforcement learning based on the feedback.

17. The method of claim 10 , further comprising providing a user interface that allows a human operator to override operation of the vehicle by the artificial intelligence system.

18. The method of claim 10 , wherein the vehicle is a self-driving vehicle and the human interactions are simulated based on data from a database of human driving behavior.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 24, 2024
From: CELLA, CHARLES HOWARD
To: STRONG FORCE TP PORTFOLIO 2022, LLC
Reel/Frame 066235/0543 →
Continuity (5)
Continuation 17977791 · Oct 31, 2022
Continuation 16803356 · Feb 27, 2020
Continuation PCTUS2019053857 · Sep 30, 2019
Provisional Application 62739335 · Sep 30, 2018
Related Publication 20240126288A1 · Apr 18, 2024
References Cited (49)
US 5400734A · Doyon · 1995 [cited by applicant]
US 5566072A · Momose et al. · 1996 [cited by applicant]
US 6058352A · Lu et al. · 2000 [cited by applicant]
US 6188945B1 · Graf et al. · 2001 [cited by applicant]
US 7982620B2 · Prokhorov et al. · 2011 [cited by applicant]
US 9451396B2 · Takatsuji et al. · 2016 [cited by applicant]
US 9744363B2 · Grill, Jr. et al. · 2017 [cited by applicant]
US 9827993B2 · Prokhorov · 2017 [cited by applicant]
US 9889770B2 · Hotary et al. · 2018 [cited by applicant]
US 9914475B2 · Sun et al. · 2018 [cited by applicant]
US 10117109B2 · Madaiah et al. · 2018 [cited by applicant]
US 10122692B2 · MacCarthaigh · 2018 [cited by applicant]
US 10126743B2 · Fukumoto · 2018 [cited by applicant]
US 10289618B2 · Fink et al. · 2019 [cited by applicant]
US 10346415B1 · Smith et al. · 2019 [cited by applicant]
US 10373522B2 · Byron et al. · 2019 [cited by applicant]
US 10387205B2 · Boutnaru · 2019 [cited by applicant]
US 10387417B1 · Hansen et al. · 2019 [cited by applicant]
US 10387429B2 · Smith-Mickelson et al. · 2019 [cited by applicant]
US 10482333B1 · El Kaliouby et al. · 2019 [cited by applicant]
US 10691943B1 · Ferstl et al. · 2020 [cited by applicant]
US 10850693B1 · Pertsel et al. · 2020 [cited by applicant]
US 10894545B2 · Bielby · 2021 [cited by examiner]
US 20050143845A1 · Kaji · 2005 [cited by applicant]
US 20090292528A1 · Kameyama · 2009 [cited by applicant]
US 20110213511A1 · Visconti et al. · 2011 [cited by applicant]
US 20150166069A1 · Engelman et al. · 2015 [cited by applicant]
US 20150301602A1 · Dow et al. · 2015 [cited by applicant]
US 20150344036A1 · Kristinsson et al. · 2015 [cited by applicant]
US 20160189056A1 · Mayhew · 2016 [cited by applicant]
US 20170285642A1 · Rander · 2017 [cited by applicant]
US 20170299399A1 · Yamaguchi et al. · 2017 [cited by applicant]
US 20180113461A1 · Potnis et al. · 2018 [cited by applicant]
US 20180260414A1 · Gordo Soldevila · 2018 [cited by applicant]
US 20190035394A1 · Tsai · 2019 [cited by applicant]
US 20190049957A1 · Healey et al. · 2019 [cited by applicant]
US 20190049969A1 · Qawami et al. · 2019 [cited by applicant]
US 20190197430A1 · Arditi · 2019 [cited by applicant]
US 20190265060A1 · Han et al. · 2019 [cited by applicant]
US 20200206631A1 · Sumant et al. · 2020 [cited by applicant]
US 20200215294A1 · Lee et al. · 2020 [cited by applicant]
US 20210354726A1 · Bill et al. · 2021 [cited by applicant]
CN 107364368A · 2017 [cited by applicant]
KR 20150050134A · 2015 [cited by applicant]
WO 2009061687A1 · 2009 [cited by applicant]
Bojarksi et al., “End to End Learning for Self-Driving Cars”, arXiv preprint arXiv: 1604.07316, Apr. 25, 2016. [cited by applicant]
Author Unknown, “The Difference Between Robotic Process Automation and Artificial Intelligence”, CFB Bots, retrieved from https://medium.com/@cfb_bots, 7 pages, Apr. 20, 2018. [cited by applicant]
Popa, Alexandra, et al., “Intelligent Autonomous Driving”, 2018 22nd International Conference on System Theory, Control and Computing (ICSTCC), pp. 53-58, 2018. [cited by applicant]
Kocic et al., “Sim-to-Real Autonomous Vehicle Lane Keeping using Vision”, 2021 29th Telecommunications Forum (TELFOR), pp. 1-8, 2021. [cited by applicant]
Cited By (5)
US 12,393,915 US 12,498,680 US 12,585,231 US 12,585,654 US 12,589,498