IP Library › Granted Patent US 12,516,501
Granted Patent B2
US 12,516,501 · App. 18/302,429 · Granted Jan 6, 2026

Trajectory controller for continuous track vehicles

Inventors: Ruitao Song (Sunnyvale, CA); Liangjun Zhang (Sunnyvale, CA)
Assignee: Baidu USA LLC
E02F9/2045E02F9/205E02F9/26G06N7/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,516,501
App. No.
18/302,429
Granted
Jan 6, 2026
Kind
B2
Abstract

Embodiments of a methodology for controlling a vehicle includes (i) determining a first command signal for a first locomotion component of a vehicle and a second command signal for a second locomotion component of the vehicle, (ii) based upon a terrain classification, selecting a first pre-trained model for the first locomotion component and a second pre-trained model for the second locomotion component, (iii) determining a first signal for the first locomotion component of the vehicle by utilizing the first command signal and the second command signal as input to the first pre-trained model and a second signal for the second locomotion component of the vehicle by utilizing the first command signal and the second command signal as input to the second pre-trained model, and (iv) controlling the first locomotion component of the vehicle using the first signal and the second locomotion component of the vehicle using the second signal.

Claims (37)

1 . A computer-implemented method for controlling a trajectory of a continuous track vehicle based upon vehicle trajectory input, comprising:

determining, via a controller, (i) a first command signal for a first locomotion component of a vehicle and (ii) a second command signal for a second locomotion component of the vehicle;

based upon a terrain classification, selecting (i) a first pre-trained model for the first locomotion component of the vehicle and (ii) a second pre-trained model for the second locomotion component of the vehicle;

determining (i) a first signal for the first locomotion component of the vehicle by utilizing the first command signal and the second command signal as input to the first pre-trained model and (ii) a second signal for the second locomotion component of the vehicle by utilizing the first command signal and the second command signal as input to the second pre-trained model; and

controlling (i) the first locomotion component of the vehicle using the first signal and (ii) the second locomotion component of the vehicle using the second signal;

wherein the controller comprises a nonlinear model predictive controller that determines the first command signal and the second command signal based upon vehicle state input and based upon the vehicle trajectory input; and

wherein the first locomotion component and the second locomotion component each comprise continuous track locomotion components.

2 . The computer-implemented method of claim 1 , wherein the first signal comprises a first pulse-width modulated signal, and wherein the second signal comprises a second pulse-width modulated signal.

3 . The computer-implemented method of claim 1 , wherein the terrain classification is obtained from a terrain classification model that uses image data, inertial measurement unit data, or both as inputs.

4 . The computer-implemented method of claim 3 , wherein selecting the first pre-trained model and the second pre-trained model is further based upon the image data or the inertial measurement data.

5 . The computer-implemented method of claim 1 , wherein the first model and the second model comprise a set of Gaussian Process Models (GPMs) in which each GPM is correlated to a terrain type.

6 . A computer-implemented method for facilitating trajectory control of a continuous track vehicle based upon vehicle trajectory input, comprising:

receiving a terrain classification;

accessing a set of pre-trained models, each pre-trained model of the set of pre-trained models being configured to determine a signal in response to input command signals, the set of pre-trained models comprising a plurality of subsets of pre-trained models, wherein each subset of pre-trained models is associated with a respective known terrain;

each subset of pre-trained models comprising a respective first pre-trained model of the subset and a respective second pre-trained model of the subset trained using trajectory data associated with one or more vehicles traversing the respective known terrain, wherein each respective first pre-trained model of the subset is associated with a first locomotion component that is associated with the one or more vehicles, and wherein each respective second pre-trained model of the subset is associated with a second locomotion component that is associated with the one or more vehicles;

selecting a particular subset of pre-trained models from the plurality of subsets of pre-trained models based upon at least the terrain classification;

receiving (i) a first command signal associated with the first locomotion component and (ii) a second command signal associated with the second locomotion component; and

determining (i) a first signal based on the respective first pre-trained model of the selected subset of pre-trained models and (ii) a second signal based on the respective second pre-trained model of the selected subset of pre-trained models; and

controlling the first locomotion component based on the determined first signal and controlling the second locomotion component based on the determined second signal;

wherein the first command signal and the second command signal are obtained from a nonlinear model predictive controller; and

wherein the nonlinear model predictive controller determines the first command signal and the second command signal based upon vehicle state input and based upon the vehicle trajectory input; and

wherein the first locomotion component and the second locomotion component each comprise continuous track locomotion components.

7 . The computer-implemented method of claim 6 , wherein the first pre-trained model utilizes both the first command signal and the second command signal as input to determine the first signal, and wherein the second pre-trained model utilizes both the first command signal and the second command signal as input to determine the second signal.

8 . The computer-implemented method of claim 6 , wherein the set of pre-trained models comprises a set of Gaussian Process Models (GPMs).

9 . The computer-implemented method of claim 6 , wherein the first signal comprises a first pulse-width modulated signal, and wherein the second signal comprises a second pulse-width modulated signal.

10 . The computer-implemented method of claim 6 , wherein the terrain classification is obtained from a terrain classification model that uses image data, inertial measurement data, or both as inputs.

11 . The computer-implemented method of claim 10 , wherein selecting the particular subset of pre-trained models is further based upon the image data or the inertial measurement data.

12 . A system for controlling a trajectory of a continuous track vehicle based upon vehicle trajectory input, comprising:

one or more processors; and

a non-transitory computer-readable medium or media comprising one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:

determining, via a controller, (i) a first command signal for a first locomotion component of a vehicle and (ii) a second command signal for a second locomotion component of the vehicle;

selecting (i) a first pre-trained model for the first locomotion component of the vehicle and (ii) a second pre-trained model for the second locomotion component of the vehicle;

determining (i) a first signal for the first locomotion component of the vehicle by utilizing the first command signal and the second command signal as input to the first pre-trained model and (ii) a second signal for the second locomotion component of the vehicle by utilizing the first command signal and the second command signal as input to the second pre-trained model; and

controlling (i) the first locomotion component of the vehicle using the first signal and (ii) the second locomotion component of the vehicle using the second signal;

wherein the controller comprises a nonlinear model predictive controller that determines the first command signal and the second command signal based upon vehicle state input and the vehicle trajectory input; and

wherein the first locomotion component and the second locomotion component each comprise continuous track locomotion components.

13 . The system of claim 12 , wherein the first pre-trained model and the second pre-trained model are selected based upon a terrain classification, and wherein the terrain classification is obtained from a terrain classification model that uses image data, inertial measurement unit data, or both as inputs.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 29, 2023
From: SONG, RUITAO; ZHANG, LIANGJUN
To: BAIDU USA, LLC
Reel/Frame 064742/0656 →
Continuity (1)
Related Publication 20240352708A1 · Oct 24, 2024
References Cited (101)
US 5229944A · Yasuno · 1993 [cited by examiner]
US 5590041A · Cooper · 1996 [cited by examiner]
US 6076030A · Rowe · 2000 [cited by examiner]
US 9376108B2 · Fairgrieve · 2016 [cited by examiner]
US 9428184B2 · Kikuchi · 2016 [cited by examiner]
US 9965868B2 · Liu · 2018 [cited by examiner]
US 11409287B2 · Zhang · 2022 [cited by examiner]
US 11543831B2 · Reid · 2023 [cited by examiner]
US 12280796B1 · Schleede · 2025 [cited by examiner]
US 12286151B1 · Greiff · 2025 [cited by examiner]
US 20050183416A1 · Hayashi · 2005 [cited by examiner]
US 20080086248A1 · Lu · 2008 [cited by examiner]
US 20080133066A1 · Takenaka · 2008 [cited by examiner]
US 20090088918A1 · Takenaka · 2009 [cited by examiner]
US 20120179635A1 · Vasudevan · 2012 [cited by examiner]
US 20130313033A1 · Elliott · 2013 [cited by examiner]
US 20140032046A1 · Meffert · 2014 [cited by examiner]
US 20140156152A1 · Faivre · 2014 [cited by examiner]
US 20140257621A1 · Zych · 2014 [cited by examiner]
US 20150112508A1 · Kato · 2015 [cited by examiner]
US 20150275482A1 · Hill · 2015 [cited by examiner]
US 20170113697A1 · Faivre · 2017 [cited by examiner]
US 20180043932A1 · Borud · 2018 [cited by examiner]
US 20190113924A1 · Falconer · 2019 [cited by examiner]
US 20190155229A1 · Herrera · 2019 [cited by examiner]
US 20190338490A1 · Roulston · 2019 [cited by examiner]
US 20200094843A1 · Bärecke · 2020 [cited by examiner]
US 20200290625A1 · Berntorp · 2020 [cited by examiner]
US 20200293057A1 · Reid · 2020 [cited by examiner]
US 20200331317A1 · Nasu · 2020 [cited by examiner]
US 20210062472A1 · Daimon · 2021 [cited by examiner]
US 20210078592A1 · Febbo · 2021 [cited by examiner]
US 20210140145A1 · Daimon · 2021 [cited by examiner]
US 20210206430A1 · North · 2021 [cited by examiner]
US 20210223774A1 · Zhang · 2021 [cited by examiner]
US 20210264300A1 · Staudinger · 2021 [cited by examiner]
US 20210310216A1 · Daimon · 2021 [cited by examiner]
US 20210323156A1 · Couvignou · 2021 [cited by examiner]
US 20220081878A1 · Phadtare · 2022 [cited by examiner]
US 20220111736A1 · Takeuchi · 2022 [cited by examiner]
US 20220307235A1 · Taylor · 2022 [cited by examiner]
US 20220325502A1 · Hobbs · 2022 [cited by examiner]
US 20220350991A1 · Vesperman · 2022 [cited by examiner]
US 20230038422A1 · Redden · 2023 [cited by examiner]
US 20230046970A1 · Li · 2023 [cited by examiner]
US 20230053964A1 · Wang · 2023 [cited by examiner]
US 20230100858A1 · Hirao · 2023 [cited by examiner]
US 20230166722A1 · Hajiloo · 2023 [cited by examiner]
US 20230286523A1 · Dumas · 2023 [cited by examiner]
US 20230313490A1 · Anderson · 2023 [cited by examiner]
US 20240116565A1 · Subramanian · 2024 [cited by examiner]
US 20240131705A1 · Ye · 2024 [cited by examiner]
US 20240270084A1 · Takahashi · 2024 [cited by examiner]
US 20240352708A1 · Song · 2024 [cited by examiner]
US 20240370016A1 · Lu · 2024 [cited by examiner]
US 20250018950A1 · Oh · 2025 [cited by examiner]
US 20250163681A1 · Gajic · 2025 [cited by examiner]
US 20250171983A1 · Gajic · 2025 [cited by examiner]
US 20250198126A1 · Hobbs · 2025 [cited by examiner]
US 20250313094A1 · Sadeghi Kati · 2025 [cited by examiner]
US 20250313229A1 · Davins · 2025 [cited by examiner]
M. Z. Hashmi et al., “What Lies Beneath One's Feet? Terrain Classification Using Inertial Data of Human Walk,” Applied Sciences, vol. 9, 2019. (24pgs). [cited by applicant]
F. Lomio et al., “Surface type classification for autonomous robot indoor navigation,” arXiv preprint arXiv:1905.00252, 2019. (5 pgs). [cited by applicant]
D. Khan et al., “Recent advances in vision-based indoor navigation:A systematic literature review,” Computers & Graphics, 2022. [Abstract] (9pgs). [cited by applicant]
A. J. Sathyamoorthy et al., “DenseCAvoid: Real-time Navigation in Dense Crowds using Anticipatory Behaviors,” arXiv preprint arXiv:2002.03038, 2020. (8pgs). [cited by applicant]
A. J. Sathyamoorthy et al., “Frozone: Freezing-free, pedestrian-friendly navigation in human crowds,” arXiv preprint arXiv:2003.05395, 2020. (9pgs). [cited by applicant]
D. Droeschel et al., “Continuous Mapping & Localization for Autonomous Navigation in Rough Terrain using a 3D Laser Scanner” Robotics & Autonomous Systems, vol. 88, 2017.(20pg). [cited by applicant]
K. Weerakoon et al., “TERP: Reliable Planning in Uneven Outdoor Environments using Deep Reinforcement Learning,” arXiv preprint arXiv:2109.05120, 2022. (8pgs). [cited by applicant]
J. Liang et al., “AdaptiveON: Adaptive Outdoor Local Navigation Method for Stable and Reliable Actions,” arXiv preprint arXiv:2205.03517, 2022. (10pgs). [cited by applicant]
A. Kumar et al., “RMA: Rapid Motor Adaptation for Legged Robots,” arXiv preprint arXiv:2107.04034, 2021. (15pgs). [cited by applicant]
H. Karnan et al., “VI-IKD: High-Speed Accurate Off-Road Navigation using Learned Visual-Inertial Inverse Kinodynamics,” arXiv preprint arXiv:2203.15983, 2022. (8pgs). [cited by applicant]
J. Dallas et al., “Terrain Adaptive Trajectory Planning and Tracking on Deformable Terrains,” IEEE Transactions on Vehicular Technology, vol. 70, No. 11, 2021. (14pgs). [cited by applicant]
T. Guan et al., “GA-Nav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environments,” arXiv preprint arXiv:2103.04233, 2022. (12pgs). [cited by applicant]
K. Viswanath et al., “Offseg: A Semantic Segmentation Framework for Off-Road Driving,” arXiv preprint arXiv:2103.12417, 2021. (6pgs). [cited by applicant]
A. Chilian et al., “Stereo camera based navigation of mobile robots on rough terrain,” in 2009 IEEE/RSJ International Conference on Intelligent Robots & Systems, 2009. (6pgs). [cited by applicant]
J. Xue et al., “Deep texture manifold for ground terrain recognition,” in 2018 IEEE/CVF Conference on Computer Vision & Pattern Recognition, 2018. (10pgs). [cited by applicant]
S. Laible et al., “3D LIDAR- and Camera-Based Terrain Classification Under Different Lighting Conditions,” in AMS, 2012. (9pgs). [cited by applicant]
D.-V. Nguyen, “Vegetation Detection and Terrain Classification for Autonomous Navigation,” Ph.D. dissertation, 2013. (212 pgs). [cited by applicant]
J. Xue et al., “Differential angular imaging for material recognition,” in 2017 IEEE Conference on Computer Vision & Pattern Recognition (CVPR), 2017. (10pgs). [cited by applicant]
C. C. Ward, “Terrain Sensing and Estimation for Dynamic Outdoor Mobile Robots,” Ph.D. dissertation, Massachusetts Institute of Technology, 2007. (126pgs). [cited by applicant]
C. C. Ward et al., “Speed-independent vibration-based terrain classification for passenger vehicles,” Vehicle System Dynamics, vol. 47, 2009. [Abstract] (2pgs). [cited by applicant]
Z. Liu et al., “Swin Transformer V2: Scaling Up Capacity and Resolution,” in Proc. of the IEEE/CVF Conference on Computer Vision & Pattern Recognition (CVPR), 2022. (11pgs). [cited by applicant]
E. Xie et al., “SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers,” in Advances in Neural Information Processing Systems, 2021. (14pgs). [cited by applicant]
Y. Yuan et al., “Object-Contextual Representations for Semantic Segmentation,” in 16th European Conference Computer Vision (ECCV 2020), Aug. 2020. (18pgs). [cited by applicant]
Z. Dai et al., “CoAtNet: Marrying Convolution and Attention for All Data Sizes,” in 35th Conference on Neural Information Processing Systems (NeurIPS), 2021. (13pgs). [cited by applicant]
A. J. Sathyamoorthy et al., “TerraPN: Unstructured Terrain Navigation using Online Self-Supervised Learning,” arXiv preprint arXiv:2202.12873, 2022. (10pgs). [cited by applicant]
O. Russakovsky et al., “ImageNet Large Scale Visual Recognition Challenge,” arXiv preprint arXiv:1409.0575, 2015. (43pgs). [cited by applicant]
T.-Y. Lin et al., “Microsoft COCO: Common Objects in Context,” arXiv preprint arXiv:1405.0312, 2014. (14pgs). [cited by applicant]
T. Guan et al., “TNS: Terrain Traversability Mapping and Navigation System for Autonomous Excavators,” arXiv preprint arXiv:2109.06250, 2022. (15pgs). [cited by applicant]
C. D. McKinnon et al., “Experience-Based Model Selection to Enable Long-Term, Safe Control for Repetitive Tasks Under Changing Conditions,” In: IEEE/RSJ Int. Conf. on Intelligent Robots & Systems (IROS), IEEE, 2018. (8p… [cited by applicant]
C. H. Lee et al., “Double-track mobile robot for hazardous environment applications,” Advanced Robotics, vol. 17, No. 5, 2003 [Abstract]. (2pgs). [cited by applicant]
Vincent et al., “A combined reactive and reinforcement learning controller for an autonomous tracked vehicle,” Robotics & Autonomous Systems, vol. 60, No. 4,2012 [Abstract]. (6pgs). [cited by applicant]
C. Zong et al., “An angle-changeable tracked robot with human-robot interaction in unstructured environments,” Assembly Automation, vol. 40, No. 4, 2020 [Abstract]. (2pgs). [cited by applicant]
C. Ordonez et al., “Modeling and traversal of pliable materials for tracked robot navigation,” in Unmanned Systems Technology XX, vol. 10640. SPIE, 2018 [Abstract]. (3pgs). [cited by applicant]
L. Hewing et al., “Learning-based model predictive control: Toward safe learning in control,” Annual Review of Control, Robotics, & Autonomous Systems, vol. 3, 2020. (269pgs). [cited by applicant]
G. Torrente et al., “Data-Driven MPC for Quadrotors,” arXiv preprint arXiv: 2102.05773, 2021. (8pgs). [cited by applicant]
A. H. Chang et al., “Every Hop is an Opportunity: Quickly Classifying and Adapting to Terrain During Targeted Hopping,” In: International Conference on Robotics & Automation (ICRA), 2019. (7pgs). [cited by applicant]
M. Tan et al., “EfficientNet: Rethinking model scaling for convolutional neural networks,” arXiv preprint arXiv:1905.11946, 2019. (10pgs). [cited by applicant]
A. Vaswani et al., “Attention is all you Need,” arXiv preprint arXiv:1706.03762, 2017. (15pgs). [cited by applicant]
T. Guan et al., “VINet: Visual and Inertial-based Terrain Classification and Adaptive Navigation over Unknown Terrain,” arXiv preprint arXiv:2209.07725, 2022. (7pgs). [cited by applicant]
A. Kurup et al., “Supervised terrain classification with adaptive unsupervised terrain assessment,” In: SAE International Journal of Advances & Current Practices in Mobility, 2021. (9pgs). [cited by applicant]