IP Library Granted Patent US 10,703,370
Granted Patent B2
US 10,703,370 · App. 16/111,504 · Granted Jul 7, 2020

Vehicle action control

Inventors: Arpan Kusari (East Lansing, MI); Hongtei Eric Tseng (Canton, MI)
Assignee: FORD GLOBAL TECHNOLOGIES, LLC
B60W30/18163G05D1/0217G05D1/0221G05D1/0223B60W30/09B60W30/0956B60W30/16B60W2420/42B60W2420/52G05D1/0088G05D1/0214G08G1/167
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,703,370
App. No.
16/111,504
Granted
Jul 7, 2020
Kind
B2
Abstract

One or more target areas are identified proximate to a moving vehicle. The vehicle can be maneuvered to a target area selected according to a reinforcement learning reward function.

Claims (29)

1. A computer comprising a processor and a memory, the memory storing instructions executable by the processor to:

identify one or more target areas proximate to a moving vehicle; and

maneuver the moving vehicle to a target area selected according to a reinforcement learning reward function;

wherein the reward function includes a difference between a current speed and a speed that would result from taking an action affecting movement of the moving vehicle, an amount of time that the moving vehicle has operated without collision in a simulation episode, and a penalty assessed if a lane change results from the action.

2. The computer of claim 1 , wherein the reward function includes a penalty for a safety risk.

3. The computer of claim 1 , wherein the reward function evaluates a relative velocity of a second vehicle to specify a reward.

4. The computer of claim 1 , wherein the reward function evaluates a length of the target area to specify a reward.

5. The computer of claim 1 , wherein the one or more target areas are specified according to a maximum longitudinal distance between vehicles on a roadway.

6. The computer of claim 1 , wherein the one or more target areas are specified according to a boundary of at least one second vehicle.

7. The computer of claim 1 , wherein the one or more target areas include target areas in a plurality of lanes on a roadway.

8. The computer of claim 1 , wherein the instructions further include instructions to determine that the moving vehicle is moving above a predetermined velocity threshold prior to selecting the target area.

9. A method, comprising:

identifying one or more target areas proximate to a moving vehicle; and

maneuvering the moving vehicle to a target area selected according to a reinforcement learning reward function;

wherein the reward function includes a difference between a current speed and a speed that would result from taking an action affecting movement of the moving vehicle, an amount of time that the moving vehicle has operated without collision in a simulation episode, and a penalty assessed if a lane change results from the action.

10. The method of claim 9 , wherein the reward function includes a penalty for a safety risk.

11. The method of claim 9 , wherein the reward function evaluates a relative velocity of a second vehicle to specify a reward.

12. The method of claim 9 , wherein the reward function evaluates a length of the target area to specify a reward.

13. The method of claim 9 , wherein the one or more target areas are specified according to a maximum longitudinal distance between vehicles on a roadway.

14. The method of claim 9 , wherein the one or more target areas are specified according to a boundary of at least one second vehicle.

15. The method of claim 9 , wherein the one or more target areas include target areas in a plurality of lanes on a roadway.

16. The method of claim 9 , further comprising determining that the moving vehicle is moving above a predetermined velocity threshold prior to selecting the target area.

17. A vehicle, comprising:

a sensor providing data about an area around the vehicle; and

a computer comprising a processor and a memory, the memory storing instructions executable by the processor to:

based on data from the sensor, identify one or more target areas proximate to a moving vehicle; and

maneuver the vehicle to a target area selected according to a reinforcement learning reward function;

wherein the reward function includes a difference between a current speed and a speed that would result from taking an action affecting movement of the moving vehicle, an amount of time that the moving vehicle has operated without collision in a simulation episode, and a penalty assessed if a lane change results from the action.

18. The vehicle of claim 17 , wherein the one or more target areas include target areas in a plurality of lanes on a roadway.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 24, 2018
From: KUSARI, ARPAN; TSENG, HONGTEI ERIC
To: FORD GLOBAL TECHNOLOGIES, LLC
Reel/Frame 046694/0079 →
Continuity (1)
Related Publication 20200062262A1 · Feb 27, 2020
Cited By (1)
US 12,371,025