IP Library Granted Patent US 12,697,978
Granted Patent B2
US 12,697,978 · App. 18/205,944 · Granted Aug 4, 2026

System and method for path planning of autonomous vehicles based on gradient

Inventors: Wutu Lin (San Diego, CA); Xiaodi Hou (San Diego, CA)
Assignee: CreateAI, Inc.
B60W40/04B60W30/0956B60W60/0011G01C21/20G01S13/931G01S17/931G01S13/865G01S13/867
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,697,978
App. No.
18/205,944
Filed
Jun 5, 2023
Granted
Aug 4, 2026
Kind
B2
Art Unit
3665
USPC
701/23
Abstract

A system and method for path planning of autonomous vehicles based on gradient are disclosed. A particular embodiment includes: executing a path planning module by use of a data processor; receiving, from a remote data source by use of the data processor, ancillary data to modify operation of the path planning module based on a context in which an autonomous vehicle is operating; generating, by use of the data processor, a trajectory comprising a plurality of waypoints; and causing the autonomous vehicle to follow a path conforming to the plurality of waypoints.

Claims (42)

1 . A system comprising:

a data processor; and

a path planning module, executable by the data processor, the path planning module being configured to:

receive, from a remote data source, ancillary data to modify operation of the path planning module based on a context in which an autonomous vehicle is operating;

generate, by a modified operation of the path planning module, a first suggested trajectory comprising a plurality of waypoints;

generate, based on the first suggested trajectory, a trajectory gradient comprising scale and direction data corresponding to each of the plurality of waypoints;

modify, based on the trajectory gradient, parameters of a human driving model;

generate, based on the trajectory gradient and modified parameters of the human driving model, a second suggested trajectory; and

cause the autonomous vehicle to follow a path conforming to the second suggested trajectory.

2 . The system of claim 1 wherein the path planning module is configured to generate the first suggested trajectory using machine learnable components.

3 . The system of claim 1 , the trajectory gradient is generated based on a mathematical derivative of a function defining the first suggested trajectory for the autonomous vehicle.

4 . The system of claim 1 , wherein the path planning module is configured to score the first suggested trajectory.

5 . The system of claim 1 wherein the path planning module is configured to retain information corresponding to human driving behaviors as mathematical or data representations.

6 . The system of claim 1 , wherein the path planning module is configured to output information indicative of the path to a vehicle control subsystem causing the autonomous vehicle to follow the path conforming to the second suggested trajectory.

7 . The system of claim 1 wherein the remote data source is in wireless data communication with the autonomous vehicle.

8 . The system of claim 1 , wherein the path planning module is configured to:

generate a first score for the first suggested trajectory;

generate a second score for the second suggested trajectory;

determine, prior to causing the autonomous vehicle to follow the path conforming to the second suggested trajectory, that a difference between the first score and the second score is within a score differential threshold.

9 . A method comprising:

executing a path planning module by use of a data processor;

receiving, from a remote data source by use of the data processor, ancillary data to modify operation of the path planning module based on a context in which an autonomous vehicle is operating;

generating, by use of the data processor, a first suggested trajectory comprising a plurality of waypoints;

generating, by use of the data processor, based on the first suggested trajectory, a trajectory gradient comprising scale and direction data corresponding to each of the plurality of waypoints;

modifying, based on the trajectory gradient, parameters of a human driving model;

generating, based on the trajectory gradient and modified parameters of the human driving model, a second suggested trajectory; and

causing the autonomous vehicle to follow a path conforming to the second suggested trajectory.

10 . The method of claim 9 , wherein the human driving model is trained with training data corresponding to human driving behaviors.

11 . The method of claim 9 , wherein the trajectory gradient is generated based on a mathematical derivative of a function defining the first suggested trajectory for the autonomous vehicle.

12 . The method of claim 9 , including scoring the first suggested trajectory.

13 . The method of claim 9 , including causing the autonomous vehicle to follow a different path if a score corresponding to the first suggested trajectory is not within a score differential threshold.

14 . A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:

execute a path planning module by use of a data processor;

receive, from a remote data source, ancillary data to modify operation of the path planning module based on a context in which an autonomous vehicle is operating;

generate, by a modified operation of the path planning module, a first suggested trajectory comprising a plurality of waypoints;

generate, based on the first suggested trajectory, a trajectory gradient comprising scale and direction data corresponding to each of the plurality of waypoints;

modify, based on the trajectory gradient, parameters of a human driving model;

generate, based on the trajectory gradient and modified parameters of the human driving model, a second suggested trajectory; and

cause the autonomous vehicle to follow a path conforming to the second suggested trajectory.

15 . The non-transitory machine-useable storage medium of claim 14 , wherein the trajectory gradient is generated based on a mathematical derivative of a function defining the first suggested trajectory for the autonomous vehicle.

16 . The non-transitory machine-useable storage medium of claim 14 , wherein the human driving model is trained with training data corresponding to human driving behaviors.

17 . The non-transitory machine-useable storage medium of claim 14 , wherein the path planning module is configured to output information indicative of the path to a vehicle control subsystem causing the autonomous vehicle to follow the path conforming to the second suggested trajectory.

Assignments (3)
CHANGE OF NAME Recorded Dec 3, 2025
From: TUSIMPLE, INC.
To: CREATEAI, INC.
Reel/Frame 073832/0485 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 19, 2023
From: LIN, WUTU; HOU, XIAODI
To: TUSIMPLE
Reel/Frame 064947/0170 →
CHANGE OF NAME Recorded Sep 19, 2023
From: TUSIMPLE
To: TUSIMPLE, INC.
Reel/Frame 064947/0450 →
Continuity (3)
Continuation 16900611 · Jun 12, 2020
Continuation 15481877 · Apr 7, 2017
Related Publication 20230391340A1 · Dec 7, 2023
References Cited (173)
US 6777904B1 · Degner · 2004 [cited by applicant]
US 7103460B1 · Breed · 2006 [cited by applicant]
US 7689559B2 · Canright · 2010 [cited by applicant]
US 7783403B2 · Breed · 2010 [cited by applicant]
US 7844595B2 · Canright · 2010 [cited by applicant]
US 8041111B1 · Wilensky · 2011 [cited by applicant]
US 8064643B2 · Stein · 2011 [cited by applicant]
US 8082101B2 · Stein · 2011 [cited by applicant]
US 8164628B2 · Stein · 2012 [cited by applicant]
US 8175376B2 · Marchesotti · 2012 [cited by applicant]
US 8271871B2 · Marchesotti · 2012 [cited by applicant]
US 8378851B2 · Stein · 2013 [cited by applicant]
US 8392117B2 · Dolgov · 2013 [cited by applicant]
US 8401292B2 · Park · 2013 [cited by applicant]
US 8412449B2 · Trepagnier · 2013 [cited by applicant]
US 8478072B2 · Aisaka · 2013 [cited by applicant]
US 8553088B2 · Stein · 2013 [cited by applicant]
US 8788134B1 · Litkouhi · 2014 [cited by applicant]
US 8908041B2 · Stein · 2014 [cited by applicant]
US 8917169B2 · Schofield · 2014 [cited by applicant]
US 8963913B2 · Baek · 2015 [cited by applicant]
US 8965621B1 · Urmson · 2015 [cited by applicant]
US 8981966B2 · Stein · 2015 [cited by applicant]
US 8993951B2 · Schofield · 2015 [cited by applicant]
US 9002632B1 · Emigh · 2015 [cited by applicant]
US 9008369B2 · Schofield · 2015 [cited by applicant]
US 9025880B2 · Perazzi · 2015 [cited by applicant]
US 9042648B2 · Wang · 2015 [cited by applicant]
US 9111444B2 · Kaganovich · 2015 [cited by applicant]
US 9117133B2 · Barnes · 2015 [cited by applicant]
US 9118816B2 · Stein · 2015 [cited by applicant]
US 9120485B1 · Dolgov · 2015 [cited by applicant]
US 9122954B2 · Srebnik · 2015 [cited by applicant]
US 9134402B2 · Sebastian · 2015 [cited by applicant]
US 9145116B2 · Clarke · 2015 [cited by applicant]
US 9147255B1 · Zhang · 2015 [cited by applicant]
US 9156473B2 · Clarke · 2015 [cited by applicant]
US 9176006B2 · Stein · 2015 [cited by applicant]
US 9179072B2 · Stein · 2015 [cited by applicant]
US 9183447B1 · Gdalyahu · 2015 [cited by applicant]
US 9185360B2 · Stein · 2015 [cited by applicant]
US 9191634B2 · Schofield · 2015 [cited by applicant]
US 9233659B2 · Rosenbaum · 2016 [cited by applicant]
US 9233688B2 · Clarke · 2016 [cited by applicant]
US 9248832B2 · Huberman · 2016 [cited by applicant]
US 9248835B2 · Tanzmeister · 2016 [cited by applicant]
US 9251708B2 · Rosenbaum · 2016 [cited by applicant]
US 9277132B2 · Berberian · 2016 [cited by applicant]
US 9280711B2 · Stein · 2016 [cited by applicant]
US 9286522B2 · Stein · 2016 [cited by applicant]
US 9297641B2 · Stein · 2016 [cited by applicant]
US 9299004B2 · Lin · 2016 [cited by applicant]
US 9315192B1 · Zhu · 2016 [cited by applicant]
US 9317033B2 · Ibanez-guzman · 2016 [cited by applicant]
US 9317776B1 · Honda · 2016 [cited by applicant]
US 9330334B2 · Lin · 2016 [cited by applicant]
US 9342074B2 · Dolgov · 2016 [cited by applicant]
US 9355635B2 · Gao · 2016 [cited by applicant]
US 9365214B2 · Ben Shalom · 2016 [cited by applicant]
US 9399397B2 · Mizutani · 2016 [cited by applicant]
US 9428192B2 · Schofield · 2016 [cited by applicant]
US 9436880B2 · Bos · 2016 [cited by applicant]
US 9438878B2 · Niebla · 2016 [cited by applicant]
US 9443163B2 · Springer · 2016 [cited by applicant]
US 9446765B2 · Ben Shalom · 2016 [cited by applicant]
US 9459515B2 · Stein · 2016 [cited by applicant]
US 9466006B2 · Duan · 2016 [cited by applicant]
US 9476970B1 · Fairfield · 2016 [cited by applicant]
US 9490064B2 · Hirosawa · 2016 [cited by applicant]
US 9531966B2 · Stein · 2016 [cited by applicant]
US 9535423B1 · Debreczeni · 2017 [cited by applicant]
US 9555803B2 · Pawlicki · 2017 [cited by applicant]
US 9568915B1 · Berntorp · 2017 [cited by applicant]
US 9587952B1 · Slusar · 2017 [cited by applicant]
US 9720418B2 · Stenneth · 2017 [cited by applicant]
US 9723097B2 · Harris · 2017 [cited by applicant]
US 9723099B2 · Chen · 2017 [cited by applicant]
US 9738280B2 · Rayes · 2017 [cited by applicant]
US 9746550B2 · Nath · 2017 [cited by applicant]
US 10710592B2 · Lin · 2020 [cited by applicant]
US 10956855B1 · Coughran · 2021 [cited by applicant]
US 11673557B2 · Lin · 2023 [cited by examiner]
US 20070230792A1 · Shashua · 2007 [cited by applicant]
US 20080249667A1 · Horvitz · 2008 [cited by applicant]
US 20090040054A1 · Wang · 2009 [cited by applicant]
US 20100049397A1 · Lin · 2010 [cited by applicant]
US 20100226564A1 · Marchesotti · 2010 [cited by applicant]
US 20100281361A1 · Marchesotti · 2010 [cited by applicant]
US 20110082637A1 · Green · 2011 [cited by examiner]
US 20110206282A1 · Aisaka · 2011 [cited by applicant]
US 20120105639A1 · Stein · 2012 [cited by applicant]
US 20120140076A1 · Rosenbaum · 2012 [cited by applicant]
US 20120274629A1 · Baek · 2012 [cited by applicant]
US 20130151107A1 · Nikovski · 2013 [cited by applicant]
US 20140145516A1 · Hirosawa · 2014 [cited by applicant]
US 20140198184A1 · Stein · 2014 [cited by applicant]
US 20150062304A1 · Stein · 2015 [cited by applicant]
US 20150353082A1 · Lee · 2015 [cited by applicant]
US 20160037064A1 · Stein · 2016 [cited by applicant]
US 20160094774A1 · Li · 2016 [cited by applicant]
US 20160129907A1 · Kim · 2016 [cited by applicant]
US 20160165157A1 · Stein · 2016 [cited by applicant]
US 20160210528A1 · Duan · 2016 [cited by applicant]
US 20160313133A1 · Zeng · 2016 [cited by applicant]
US 20160321381A1 · English · 2016 [cited by applicant]
US 20160375907A1 · Erban · 2016 [cited by applicant]
US 20180120843A1 · Berntorp · 2018 [cited by applicant]
US 20180292830A1 · Kazemi · 2018 [cited by applicant]
US 20180307236A1 · Reed · 2018 [cited by examiner]
US 20190369637A1 · Shalev-Shwartz · 2019 [cited by applicant]
EP 1754179A1 · 2007 [cited by applicant]
EP 2448251A2 · 2012 [cited by applicant]
EP 2463843A2 · 2012 [cited by applicant]
EP 2463843A3 · 2013 [cited by applicant]
EP 2761249A1 · 2014 [cited by applicant]
EP 2463843B1 · 2015 [cited by applicant]
EP 2448251A3 · 2015 [cited by applicant]
EP 2946336A2 · 2015 [cited by applicant]
EP 2993654A1 · 2016 [cited by applicant]
EP 3081419A1 · 2016 [cited by applicant]
WO WO2005098739A1 · 2005 [cited by applicant]
WO WO2005098751A1 · 2005 [cited by applicant]
WO WO2005098782 · 2005 [cited by applicant]
WO WO2010109419A1 · 2010 [cited by applicant]
WO WO2013045612 · 2013 [cited by applicant]
WO WO2014111814A2 · 2014 [cited by applicant]
WO WO2014111814A3 · 2014 [cited by applicant]
WO WO2014201324 · 2014 [cited by applicant]
WO WO2015083009 · 2015 [cited by applicant]
WO WO2015103159A1 · 2015 [cited by applicant]
WO WO2015125022 · 2015 [cited by applicant]
WO WO2015186002A2 · 2015 [cited by applicant]
WO WO2015186002A3 · 2015 [cited by applicant]
WO WO2016135736 · 2016 [cited by applicant]
WO WO2017013875A1 · 2017 [cited by applicant]
Hou, Xiaodi and Zhang, Liqing, “Saliency Detection: A Spectral Residual Approach”, Computer Vision and Pattern Recognition, CVPR'07—IEEE Conference, pp. 1-8, 2007. [cited by applicant]
Hou, Xiaodi and Harel, Jonathan and Koch, Christof, “Image Signature: Highlighting Sparse Salient Regions”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, No. 1, pp. 194-201, 2012. [cited by applicant]
Hou, Xiaodi and Zhang, Liqing, “Dynamic Visual Attention: Searching For Coding Length Increments”, Advances in Neural Information Processing Systems, vol. 21, pp. 681-688, 2008. [cited by applicant]
Li, Yin and Hou, Xiaodi and Koch, Christof and Rehg, James M. and Yuille, Alan L., “The Secrets of Salient Object Segmentation”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 280-287… [cited by applicant]
Zhou, Bolei and Hou, Xiaodi and Zhang, Liqing, “A Phase Discrepancy Analysis of Object Motion”, Asian Conference on Computer Vision, pp. 225-238, Springer Berlin Heidelberg, 2010. [cited by applicant]
Hou, Xiaodi and Yuille, Alan and Koch, Christof, “Boundary Detection Benchmarking: Beyond F-Measures”, Computer Vision and Pattern Recognition, CVPR'13, vol. 2013, pp. 1-8, IEEE, 2013. [cited by applicant]
Hou, Xiaodi and Zhang, Liqing, “Color Conceptualization”, Proceedings of the 15th ACM International Conference on Multimedia, pp. 265-268, ACM, 2007. [cited by applicant]
Hou, Xiaodi and Zhang, Liqing, “Thumbnail Generation Based on Global Saliency”, Advances in Cognitive Neurodynamics, ICCN 2007, pp. 999-1003, Springer Netherlands, 2008. [cited by applicant]
Hou, Xiaodi and Yuille, Alan and Koch, Christof, “A Meta-Theory of Boundary Detection Benchmarks”, arXiv preprint arXiv:1302.5985, 2013. [cited by applicant]
Li, Yanghao and Wang, Naiyan and Shi, Jianping and Liu, Jiaying and Hou, Xiaodi, “Revisiting Batch Normalization for Practical Domain Adaptation”, arXiv preprint arXiv:1603.04779, 2016. [cited by applicant]
Li, Yanghao and Wang, Naiyan and Liu, Jiaying and Hou, Xiaodi, “Demystifying Neural Style Transfer”, arXiv preprint arXiv:1701.01036, 2017. [cited by applicant]
Hou, Xiaodi and Zhang, Liqing, “A Time-Dependent Model of Information Capacity of Visual Attention”, International Conference on Neural Information Processing, pp. 127-136, Springer Berlin Heidelberg, 2006. [cited by applicant]
Wang, Panqu and Chen, Pengfei and Yuan, Ye and Liu, Ding and Huang, Zehua and Hou, Xiaodi and Cottrell, Garrison, “Understanding Convolution for Semantic Segmentation”, arXiv preprint arXiv:1702.08502, 2017. [cited by applicant]
Li, Yanghao and Wang, Naiyan and Liu, Jiaying and Hou, Xiaodi, “Factorized Bilinear Models for Image Recognition”, arXiv preprint arXiv:1611.05709, 2016. [cited by applicant]
Hou, Xiaodi, “Computational Modeling and Psychophysics in Low and Mid-Level Vision”, California Institute of Technology, 2014. [cited by applicant]
Spinello, Luciano, Triebel, Rudolph, Siegwart, Roland, “Multiclass Multimodal Detection and Tracking in Urban Environments”, Sage Journals, vol. 29 issue: 12, pp. 1498-1515 Article first published online: Oct. 7, 2010; … [cited by applicant]
Matthew Barth, Carrie Malcolm, Theodore Younglove, and Nicole Hill, “Recent Validation Efforts for a Comprehensive Modal Emissions Model”, Transportation Research Record 1750, Paper No. 01-0326, College of Engineering, … [cited by applicant]
Kyoungho Ahn, Hesham Rakha, “The Effects of Route Choice Decisions on Vehicle Energy Consumption and Emissions”, Virginia Tech Transportation Institute, Blacksburg, VA 24061, date unknown. [cited by applicant]
Ramos, Sebastian, Gehrig, Stefan, Pinggera, Peter, Franke, Uwe, Rother, Carsten, “Detecting Unexpected Obstacles for Self-Driving Cars: Fusing Deep Learning and Geometric Modeling”, arXiv:1612.06573v1 [cs.CV] Dec. 20, 2… [cited by applicant]
Schroff, Florian, Dmitry Kalenichenko, James Philbin, (Google), “FaceNet: A Unified Embedding for Face Recognition and Clustering”, CVPR 2015. [cited by applicant]
Dai, Jifeng, Kaiming He, Jian Sun, (Microsoft Research), “Instance-aware Semantic Segmentation via Multi-task Network Cascades”, CVPR 2016. [cited by applicant]
Huval, Brody, Tao Wang, Sameep Tandon, Jeff Kiske, Will Song, Joel Pazhayampallil, Mykhaylo Andriluka, Pranav Rajpurkar, Toki Migimatsu, Royce Cheng-Yue, Fernando Mujica, Adam Coates, Andrew Y. Ng, “An Empirical Evaluat… [cited by applicant]
Tian Li, “Proposal Free Instance Segmentation Based on Instance-aware Metric”, Department of Computer Science, Cranberry-Lemon University, Pittsburgh, PA., date unknown. [cited by applicant]
Mohammad Norouzi, David J. Fleet, Ruslan Salakhutdinov, “Hamming Distance Metric Learning”, Departments of Computer Science and Statistics, University of Toronto, date unknown. [cited by applicant]
Jain, Suyong Dutt, Grauman, Kristen, “Active Image Segmentation Propagation”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, Jun. 2016. [cited by applicant]
MacAodha, Oisin, Campbell, Neill D.F., Kautz, Jan, Brostow, Gabriel J., “Hierarchical Subquery Evaluation for Active Learning on a Graph”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition… [cited by applicant]
Kendall, Alex, Gal, Yarin, “What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision”, arXiv:1703.04977v1 [cs.CV] Mar. 15, 2017. [cited by applicant]
Wei, Junqing, John M. Dolan, Bakhtiar Litkhouhi, “A Prediction- and Cost Function-Based Algorithm for Robust Autonomous Freeway Driving”, 2010 IEEE Intelligent Vehicles Symposium, University of California, San Diego, CA… [cited by applicant]
Peter Welinder, Steve Branson, Serge Belongie, Pietro Perona, “The Multidimensional Wisdom of Crowds”; http://www.vision.caltech.edu/visipedia/papers/WelinderEtalNIPS10.pdf, 2010. [cited by applicant]
Kai Yu, Yang Zhou, Da Li, Zhang Zhang, Kaiqi Huang, “Large-scale Distributed Video Parsing and Evaluation Platform”, Center for Research on Intelligent Perception and Computing, Institute of Automation, Chinese Academy … [cited by applicant]
P. Guarneri, G. Rocca and M. Gobbi, “A Neural-Network-Based Model for the Dynamic Simulation of the Tire/Suspension System While Traversing Road Irregularities,” in IEEE Transactions on Neural Networks, vol. 19, No. 9, … [cited by applicant]
C. Yang, Z. Li, R. Cui and B. Xu, “Neural Network-Based Motion Control of an Underactuated Wheeled Inverted Pendulum Model,” in IEEE Transactions on Neural Networks and Learning Systems, vol. 25, No. 11, pp. 2004-2016, … [cited by applicant]
Stephan R. Richter, Vibhav Vineet, Stefan Roth, Vladlen Koltun, “Playing for Data: Ground Truth from Computer Games”, Intel Labs, European Conference on Computer Vision (ECCV), Amsterdam, the Netherlands, 2016. [cited by applicant]
Thanos Athanasiadis, Phivos Mylonas, Yannis Avrithis, and Stefanos Kollias, “Semantic Image Segmentation and Object Labeling”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 17, No. 3, Mar. 2007. [cited by applicant]
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele, “The Cityscapes Dataset for Semantic Urban Scene Understanding”, Proceedings of… [cited by applicant]
Adhiraj Somani, Nan Ye, David Hsu, and Wee Sun Lee, “DESPOT: Online POMDP Planning with Regularization”, Department of Computer Science, National University of Singapore, date unknown. [cited by applicant]
Adam Paszke, Abhishek Chaurasia, Sangpil Kim, and Eugenio Culurciello. Enet: A deep neural network architecture for real-time semantic segmentation. CoRR, abs/1606.02147, 2016. [cited by applicant]
Szeliski, Richard, “Computer Vision: Algorithms and Applications” http://szeliski.org/Book/, 2010. [cited by applicant]