IP Library Granted Patent US 12,276,983
Granted Patent B2
US 12,276,983 · App. 18/361,194 · Granted Apr 15, 2025

Planning autonomous motion

Inventors: Andrea Censi (Somerville, MA); Emilio Frazzoli (Newton, MA); Hsun-Hsien Chang (Brookline, MA); Kostyantyn Slutskyy (Singapore, SG); Scott D. Pendleton (Singapore, SG)
Assignee: Motional AD LLC
G05D1/0088B60W40/10G05D1/0005G05D1/0212G05D1/0214G05D1/0221G05D1/0255G05D1/0257G05D1/0268G06N5/04G06N20/00B60W2710/18B60W2710/20G01C21/34
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,276,983
App. No.
18/361,194
Granted
Apr 15, 2025
Kind
B2
Abstract

Among other things, planning a motion of a machine having moving capabilities is based on strategic guidelines derived from various basic principles, such as laws, ethics, preferences, driving experiences, and road environments.

Claims (82)

1. An apparatus comprising:

a storage for instructions;

a network interface in communication with a machine having one or more autonomous motion capabilities and configured to receive data associated with one or more motion actions of the machine; and

a processor to operate according to the instructions stored in the storage to:

analyze the data associated with the one or more motion actions of the machine, the data comprising a starting position and an ending position corresponding to the one or more motion actions of the machine;

identify a plurality of candidate trajectories for the machine to traverse through an environment based on the analyzing, wherein a candidate trajectory of the plurality of candidate trajectories comprises a plurality of motion actions for the machine;

evaluate a cost of each of the plurality of the candidate trajectories for the machine, wherein evaluating the cost of a candidate trajectory of the plurality of the candidate trajectories comprises:

identifying one or more strategic guidelines that are applicable to movement of the machine in the environment, wherein a strategic guideline corresponds to at least one environmental condition and specifies a target action corresponding to the at least one environmental condition,

comparing motion factors corresponding to each of a plurality of motion actions included in the candidate trajectory to strategic factors specified in the one or more strategic guidelines,

in response to the comparing, determining deviation metrics of the plurality of motion actions from the strategic factors, and

computing the cost of the candidate trajectory by aggregating the deviation metrics;

select a candidate trajectory of the plurality of the candidate trajectories based on evaluating respective costs; and

provide data that causes the machine to move from the starting position to the ending position in accordance with a sequence of motion actions of the selected candidate trajectory.

2. The apparatus of claim 1 , wherein analyzing the data comprises at least one of:

parsing one or more texts describing a basic principle, wherein a basic principle comprises a law, a regulation, a principle of ethics, a motion environment, an operation experience, a previously inferred logical expression, or a preference, or a combination of two or more of them, or

inferring one or more conditions corresponding to the environment and one or more proper motion actions associated with the one or more conditions.

3. The apparatus of claim 1 , wherein the network interface is configured to receive sensor data from one or more sensors associated with the machine, and

wherein analyzing the data comprises:

determining one or more objects in the sensor data, the one or more objects detected by the one or more sensors in the environment; and

inferring a logical relation between at least two of the one or more objects, or between two events, or between two times associated with the at least two objects, or combinations of them.

4. The apparatus of claim 1 , wherein identifying one or more strategic guidelines comprises statistically evaluating occurrences of environmental conditions and corresponding target actions, and deviations from the corresponding target actions, and

wherein the processor is configured to adjust the one or more strategic guidelines based on the occurrences.

5. The apparatus of claim 1 , wherein the environment is associated with at least one of the starting position or the ending position, and

wherein analyzing the data associated with the one or more motion actions of the machine comprises analyzing map information of the environment.

6. The apparatus of claim 1 , wherein analyzing map information of the environment comprises:

determining one or more non-drivable regions of the environment; and

identifying the plurality of candidate trajectories by excluding the one or more non-drivable regions of the environment.

7. The apparatus of claim 1 , wherein the one or more motion actions of the machine comprises one or more of acceleration, speed maintenance, deceleration, orientation change, stoppage, adhering to traffic rules, or collision avoidance.

8. The apparatus of claim 1 , wherein evaluating the cost of each of the plurality of the candidate trajectories for the machine further comprises selecting a cost function from one of:

a binary indicator indicating if a motion action satisfies a strategic guideline or all the strategic guidelines of a priority group, or

a numerical score characterizing the degree of deviation of a motion action from a strategic guideline or from all the strategic guidelines of a priority group.

9. The apparatus of claim 1 , wherein the machine comprises a vehicle.

10. A method comprising:

receiving, by at least one processor, data indicating a starting position and an ending position corresponding to one or more motion actions of a machine;

analyzing, by the at least one processor, map information of an environment corresponding to the starting position and the ending position;

identifying, by the at least one processor, a plurality of candidate trajectories for the machine to traverse through the environment based in response to the analyzing, wherein a candidate trajectory of the plurality of candidate trajectories comprises a plurality of motion actions of the machine;

evaluating, by the at least one processor, a cost of each of the plurality of the candidate trajectories for the machine, wherein evaluating the cost of a candidate trajectory of the plurality of the candidate trajectories comprises:

identifying one or more strategic guidelines that are applicable to movement of the machine in the environment, wherein a strategic guideline corresponds to at least one environmental condition and specifies a target action corresponding to the at least one environmental condition,

comparing motion factors corresponding to each of a plurality of motion actions included in the candidate trajectory to strategic factors specified in the one or more strategic guidelines,

in response to the comparing, determining deviation metrics of the plurality of motion actions from the strategic factors, and

computing the cost of the candidate trajectory by aggregating the deviation metrics;

selecting, by the at least one processor, a candidate trajectory of the plurality of the candidate trajectories based on evaluating respective costs; and

providing, by the at least one processor, data that causes the machine to move from the starting position to the ending position in accordance with a sequence of motion actions of the selected candidate trajectory.

11. The method of claim 10 , wherein analyzing the data comprises at least one of:

parsing one or more texts describing a basic principle, wherein a basic principle comprises a law, a regulation, a principle of ethics, a motion environment, an operation experience, a previously inferred logical expression, or a preference, or a combination of two or more of them, or

inferring one or more conditions corresponding to the environment and one or more proper motion actions associated with the one or more conditions.

12. The method of claim 10 , further comprising receiving sensor data from one or more sensors associated with the machine, wherein analyzing the data comprises:

determining one or more objects in the sensor data, the one or more objects detected by the one or more sensors in the environment; and

inferring a logical relation between at least two of the one or more objects, or between two events, or between two times associated with the at least two objects, or combinations of them.

13. The method of claim 10 , wherein identifying one or more strategic guidelines comprises statistically evaluating occurrences of environmental conditions and corresponding target actions, and deviations from the corresponding target actions, and

wherein the method further comprises adjusting the one or more strategic guidelines based on the occurrences.

14. The method of claim 10 , wherein analyzing map information of the environment comprises:

determining one or more non-drivable regions of the environment; and

identifying the plurality of candidate trajectories by excluding the one or more non-drivable regions of the environment.

15. The method of claim 10 , wherein the one or more motion actions of the machine comprises one or more of acceleration, speed maintenance, deceleration, orientation change, stoppage, adhering to traffic rules, or collision avoidance.

16. The method of claim 10 , wherein evaluating the cost of each of the plurality of the candidate trajectories for the machine further comprises selecting a cost function from one of:

a binary indicator indicating if a motion action satisfies a strategic guideline or all the strategic guidelines of a priority group, or

a numerical score characterizing the degree of deviation of a motion action from a strategic guideline or from all the strategic guidelines of a priority group.

17. The method of claim 10 , wherein the machine comprises a vehicle.

18. Non-transitory machine readable media storing instructions that, when executed, cause one or more processors to perform operations comprising:

receiving data indicating a starting position and an ending position corresponding to one or more motion actions of a machine;

analyzing map information of an environment corresponding to the starting position and the ending position;

identifying a plurality of candidate trajectories for the machine to traverse through the environment based in response to the analyzing, wherein a candidate trajectory of the plurality of candidate trajectories comprises a plurality of motion actions of the machine;

evaluating a cost of each of the plurality of the candidate trajectories for the machine, wherein evaluating the cost of a candidate trajectory of the plurality of the candidate trajectories comprises:

identifying one or more strategic guidelines that are applicable to movement of the machine in the environment, wherein a strategic guideline corresponds to at least one environmental condition and specifies a target action corresponding to the at least one environmental condition,

comparing motion factors corresponding to each of a plurality of motion actions included in the candidate trajectory to strategic factors specified in the one or more strategic guidelines,

in response to the comparing, determining deviation metrics of the plurality of motion actions from the strategic factors, and

computing the cost of the candidate trajectory by aggregating the deviation metrics;

selecting a candidate trajectory of the plurality of the candidate trajectories based on evaluating respective costs; and

providing data that causes the machine to move from the starting position to the ending position in accordance with a sequence of motion actions of the selected candidate trajectory.

19. The non-transitory machine readable media of claim 18 , wherein the operations further comprise receiving sensor data from one or more sensors associated with the machine, and

wherein analyzing the data comprises:

determining one or more objects in the sensor data, the one or more objects detected by the one or more sensors in the environment; and

inferring a logical relation between at least two of the one or more objects, or between two events, or between two times associated with the at least two objects, or combinations of them.

20. The non-transitory machine readable media of claim 18 , wherein analyzing map information of the environment comprises:

determining one or more non-drivable regions of the environment; and

identifying the plurality of candidate trajectories by excluding the one or more non-drivable regions of the environment.

21. The non-transitory machine readable media of claim 18 , wherein the one or more motion actions of the machine comprises one or more of acceleration, speed maintenance, deceleration, orientation change, stoppage, adhering to traffic rules, or collision avoidance.

22. The non-transitory machine readable media of claim 18 , wherein evaluating the cost of each of the plurality of the candidate trajectories for the machine further comprises selecting a cost function from one of:

a binary indicator indicating if a motion action satisfies a strategic guideline or all the strategic guidelines of a priority group, or

a numerical score characterizing the degree of deviation of a motion action from a strategic guideline or from all the strategic guidelines of a priority group.

23. The non-transitory machine readable media of claim 18 , wherein the machine comprises a vehicle.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2023
From: CENSI, ANDREA; FRAZZOLI, EMILIO; CHANG, HSUN-HSIEN; SLUTSKYY, KOSTYANTYN; PENDLETON, SCOTT D.
To: NUTONOMY INC.
Reel/Frame 065059/0497 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2023
From: MOTIONAL AD INC
To: MOTIONAL AD LLC
Reel/Frame 065059/0689 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2023
From: NUTONOMY INC.
To: MOTIONAL AD INC
Reel/Frame 065081/0456 →
Continuity (4)
Continuation 17114393 · Dec 7, 2020
Continuation 15872614 · Jan 16, 2018
Provisional Application 62555685 · Sep 8, 2017
Related Publication 20240028035A1 · Jan 25, 2024
References Cited (90)
US 9475422B2 · Hillis et al. · 2016 [cited by applicant]
US 9507346B1 · Levinson et al. · 2016 [cited by applicant]
US 9632502B1 · Levinson et al. · 2017 [cited by applicant]
US 9633560B1 · Gao et al. · 2017 [cited by applicant]
US 9645577B1 · Frazzoli et al. · 2017 [cited by applicant]
US 9734230B2 · Sarferaz · 2017 [cited by applicant]
US 9734455B2 · Levinson et al. · 2017 [cited by applicant]
US 9884630B1 · Ross et al. · 2018 [cited by applicant]
US 9910441B2 · Levinson et al. · 2018 [cited by applicant]
US 10048683B2 · Levinson et al. · 2018 [cited by applicant]
US 10126136B2 · Iagnemma et al. · 2018 [cited by applicant]
US 10168705B2 · Kazemi · 2019 [cited by applicant]
US 10303257B2 · Hillis et al. · 2019 [cited by applicant]
US 10309792B2 · Iagnemma · 2019 [cited by examiner]
US 10331129B2 · Iagnemma et al. · 2019 [cited by applicant]
US 10435015B2 · Kong · 2019 [cited by applicant]
US 10591910B2 · Levinson et al. · 2020 [cited by applicant]
US 10671076B1 · Kobilarov et al. · 2020 [cited by applicant]
US 10860019B2 · Censi et al. · 2020 [cited by applicant]
US 11092446B2 · Iagnemma · 2021 [cited by applicant]
US 11378955B2 · Censi et al. · 2022 [cited by applicant]
US 11392120B2 · Censi et al. · 2022 [cited by applicant]
US 11429105B2 · Sorin · 2022 [cited by examiner]
US 11714413B2 · Censi et al. · 2023 [cited by applicant]
US 20030187578A1 · Nishira et al. · 2003 [cited by applicant]
US 20080215202A1 · Breed et al. · 2008 [cited by applicant]
US 20080303696A1 · Aso et al. · 2008 [cited by applicant]
US 20110205042A1 · Takemura et al. · 2011 [cited by applicant]
US 20120083947A1 · Anderson et al. · 2012 [cited by applicant]
US 20140136414A1 · Abhyanker et al. · 2014 [cited by applicant]
US 20150292894A1 · Goddard · 2015 [cited by examiner]
US 20150336502A1 · Hillis et al. · 2015 [cited by applicant]
US 20150345966A1 · Meuleau · 2015 [cited by applicant]
US 20150348112A1 · Ramanujam · 2015 [cited by examiner]
US 20160223345A1 · Thakur et al. · 2016 [cited by applicant]
US 20160358479A1 · Riedelsheimer et al. · 2016 [cited by applicant]
US 20160378110A1 · Phillips et al. · 2016 [cited by applicant]
US 20170010679A1 · Hillis et al. · 2017 [cited by applicant]
US 20170031361A1 · Olson et al. · 2017 [cited by applicant]
US 20170123419A1 · Levinson et al. · 2017 [cited by applicant]
US 20170123428A1 · Levinson et al. · 2017 [cited by applicant]
US 20170123429A1 · Levinson et al. · 2017 [cited by applicant]
US 20170124476A1 · Levinson et al. · 2017 [cited by applicant]
US 20170235848A1 · Van Dusen et al. · 2017 [cited by applicant]
US 20170277195A1 · Frazzoli · 2017 [cited by examiner]
US 20170285585A1 · Weast et al. · 2017 [cited by applicant]
US 20170356746A1 · Iagnemma · 2017 [cited by applicant]
US 20170356747A1 · Iagnemma · 2017 [cited by applicant]
US 20170356748A1 · Iagnemma · 2017 [cited by applicant]
US 20170356750A1 · Iagnemma · 2017 [cited by applicant]
US 20180088576A1 · Kong et al. · 2018 [cited by applicant]
US 20180089563A1 · Redding · 2018 [cited by applicant]
US 20180113455A1 · Iagnemma et al. · 2018 [cited by applicant]
US 20180120857A1 · Kappauf et al. · 2018 [cited by applicant]
US 20180136644A1 · Levinson et al. · 2018 [cited by applicant]
US 20180141564A1 · Ross et al. · 2018 [cited by applicant]
US 20180255062A1 · Hopen et al. · 2018 [cited by applicant]
US 20180329411A1 · Levinson et al. · 2018 [cited by applicant]
US 20180356819A1 · Mahabadi et al. · 2018 [cited by applicant]
US 20190034794A1 · Ogale et al. · 2019 [cited by applicant]
US 20190079515A1 · Censi et al. · 2019 [cited by applicant]
US 20190079516A1 · Censi et al. · 2019 [cited by applicant]
US 20190079517A1 · Censi et al. · 2019 [cited by applicant]
US 20190079527A1 · Censi et al. · 2019 [cited by applicant]
US 20190163191A1 · Sorin et al. · 2019 [cited by applicant]
US 20190235635A1 · Hillis et al. · 2019 [cited by applicant]
US 20190308620A1 · Sapp et al. · 2019 [cited by applicant]
US 20190329769A1 · Shalev-Shwartz et al. · 2019 [cited by applicant]
US 20190329771A1 · Wray et al. · 2019 [cited by applicant]
US 20190361432A1 · Levinson et al. · 2019 [cited by applicant]
US 20210163010A1 · Takabayashi et al. · 2021 [cited by applicant]
US 20210356962A1 · Censi et al. · 2021 [cited by applicant]
CN 102334151 · 2012 [cited by applicant]
CN 106338988 · 2017 [cited by applicant]
CN 106647734 · 2017 [cited by applicant]
WO WO2017079219 · 2017 [cited by applicant]
[No Author Lsited], “SAE: International Standard J3016: Taxonomgy and Definitions for Terms Related to On-Road Motor Vehicles,” J3016, Sep. 30, 2016, 30 pages. [cited by applicant]
Chen et al., “Driving Maneuvers Prediction Based Autonomous Driving Control by Deep Monte Carlo Tree Search”, IEEE Transactions on Vehicular Technology, 69(7): 7146-7158, 2020. [cited by applicant]
Florentine et al., “Pedestrian notification methods in autonomous vehicles for multi-class mobility-on-demand service.” Proceedings of the Fourth International Conference on Human Agent Interaction, Oct. 4, 2016, pp. 38… [cited by applicant]
Gerdes et al., “Implementable Ethics for Autonomous Vehicles,” Autonomous Driving, Chapter 5, May 2016, pp. 87-102. [cited by applicant]
Huang et al., “Optimal sliding mode guidance law with height deviation and terminal impact angle constraints”, IEEE Aerospace Conference, 1-9, 2015. [cited by applicant]
IEEE Global Initiative for Ethical Consideration in Artificial Intelligence and Autonomous Systems, “Ethically Aligned Design: A Vision for Prioritizing Human Wellbeing with Artificial Intelligence and Autonomous System… [cited by applicant]
Lin et al., “UAV Collision Avoidance with Varying Trigger Time”, 2020 International Conference on Unmanned Aircraft Systems (ICUAS), 1257-1264, 2020. [cited by applicant]
Oh et al., “Fuzzy Expert Rule-Based Airborne Monitoring of Ground Vehicle Behavior,” Proceedings of 2012 UKACC International Conference on Control, Sep. 3-5, 2012, 534-539. [cited by applicant]
Pendleton et al., “Autonomous golf cars for public trial of mobility-on-demand service.” Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on Sep. 28, 2018, pp. 1164-1171. [cited by applicant]
Shavel-Shwartz et al., “Avoiding a “Winter of Autonomous Driving”: On a Formal Model of Safe, Scalable, Self-driving Cars,” arXiv preprint, Aug. 17, 2017, 25 pages. [cited by applicant]
Yang et al., “Integrating the Intelligent Driver Model With the Action Point Paradigm to Enhance the Performance of Autonomous Driving, ” IEEE Access, Jun. 3, 2020, 106284-106295. [cited by applicant]
Schmid et al., “An Efficient Sampling-Based Method for Online Informative Path Planning in Unknown Environments,” IEEE Robotics and Automation Letters, Apr. 2020, 5(2):1500-1507. [cited by applicant]
Song et al., “Dynamic Path Planning for Unmanned Vehicles Based on Fuzzy Logic and Improved Ant Colony Optimization,” Special Section on Big Data Technology and Applications in Intelligent Transportation, IEEE Access, M… [cited by applicant]
Wu et al., “Trajectory Tracking Control for Four-Wheel Independent Drive Intelligence Vehicle Based on Model Predictive Control,” IEEE Access, Apr. 14, 2020, Current Version Apr. 30, 2020, vol. 8, pp. 73071-73081. [cited by applicant]