IP Library Granted Patent US 11,273,836
Granted Patent B2
US 11,273,836 · App. 15/845,294 · Granted Mar 15, 2022

Method and system for human-like driving lane planning in autonomous driving vehicles

Inventors: Hao Zheng (Saratoga, CA); David Wanqian Liu (Los Altos, CA); Timothy Patrick Daly, Jr. (San Jose, CA)
Assignee: PlusAI, Inc.
B60W40/09B60W30/12B60W30/18163G01C21/3484G01C21/3602G01C21/3658G05D1/0088G05D1/0212G05D1/0246B60W2040/089B60W2040/0872G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,273,836
App. No.
15/845,294
Granted
Mar 15, 2022
Kind
B2
Abstract

The present teaching relates to method, system, medium, and implementation of lane planning in an autonomous vehicle. Sensor data are received that capture ground images of a road the autonomous vehicle is on. Based on the sensor data, a current lane of the road that autonomous vehicle is currently occupying is detected. Lane control for the autonomous vehicle is planned based on the detected current lane and self-aware capability parameters in accordance with a driving lane control model. The self-aware capability parameters are used to predict operational capability of the autonomous vehicle with respect to a current location of the autonomous vehicle. The driving lane control model is generated based on recorded human driving data to achieve human-like lane control behavior in different scenarios.

Claims (44)

1. A method, comprising:

receiving a plurality of driving lane control models, each driving lane control model from the plurality of driving lane control models derived based on recorded human driving data associated with a plurality of people under a plurality of different driving conditions;

receiving a set of visual images derived from real time sensor data and representing a surface of a road on which an autonomous vehicle is positioned;

detecting, on-the-fly and based on the set of visual images and at least one machine learning based lane detection model, marks on the road, the marks representing a lane of the road in which the autonomous vehicle is positioned;

selecting a driving lane control model from the plurality of driving lane control models based on a current driving condition associated with a current location of the autonomous vehicle, the current driving condition determined based on self-aware capability parameters of the autonomous vehicle, the self-aware capability parameters at least one of indicative or predictive of an operational capability of the autonomous vehicle when in the current location; and

planning lane control for the autonomous vehicle, via the selected driving lane control model, to generate a lane plan for human-like lane control behavior under the current driving condition, based on the lane in which the autonomous vehicle is positioned and the self-aware capability parameters.

2. The method of claim 1 , wherein the lane plan includes at least one of a lane following plan or a lane changing plan.

3. The method of claim 1 , wherein the self-aware capability parameters include intrinsic capability parameters and extrinsic capability parameters, the intrinsic capability parameters specifying conditions internal to the autonomous vehicle that limit the operational capability of the autonomous vehicle, and the extrinsic capability parameters specifying conditions external to the autonomous vehicle that impact the operational capability of the autonomous vehicle.

4. The method of claim 1 , further comprising dynamically updating the self-aware capability parameters to reflect a scenario currently associated with the autonomous vehicle.

5. The method of claim 1 , further comprising dynamically updating the recorded human driving data such that the lane control model is adaptive.

6. The method of claim 1 , wherein the selected driving lane control model characterizes driving lane control behavior of the plurality of people when in the current driving condition.

7. The method of claim 1 , wherein the set of visual images is a first set of visual images, the method further comprising:

analyzing a second set of visual images derived from the real time sensor data, to detect a presence of a passenger within the autonomous vehicle; and

determining, in response to detecting the presence of the passenger, a membership of the passenger in the plurality of people.

8. A machine readable, non-transitory medium storing instructions to cause a machine to:

receive a plurality of driving lane control models, each driving lane control model from the plurality of driving lane control models derived based on recorded human driving data associated with a plurality of people under a plurality of different driving conditions;

receive a set of visual images derived from real time sensor data and representing a surface of a road on which an autonomous vehicle is positioned;

detect, on-the-fly and based on the set of visual images and at least one machine learning based lane detection model, marks on the road, the marks representing a lane of the road in which the autonomous vehicle is positioned;

select a driving lane control model from the plurality of driving lane control models based on a current driving condition associated with a current location of the autonomous vehicle, the current driving condition determined based on self-aware capability parameters of the autonomous vehicle, the self-aware capability parameters at least one of indicative or predictive of an operational capability of the autonomous vehicle when in the current location; and

plan lane control for the autonomous vehicle, via the selected driving lane control model, to generate a lane plan for human-like lane control behavior under the current driving condition, based on the lane in which the autonomous vehicle is positioned and the self-aware capability parameters.

9. The medium of claim 8 , wherein the lane plan includes at least one of a lane following plan or a lane changing plan.

10. The medium of claim 8 , wherein the self-aware capability parameters include intrinsic capability parameters and extrinsic capability parameters, the intrinsic capability parameters specifying conditions internal to the autonomous vehicle that limit the operational capability of the autonomous vehicle, and the extrinsic capability parameters specifying conditions external to the autonomous vehicle that impact the operational capability of the autonomous vehicle.

11. The medium of claim 8 , further comprising dynamically updating the self-aware capability parameters to reflect a scenario currently associated with the autonomous vehicle.

12. The medium of claim 8 , further comprising dynamically updating the recorded human driving data such that the lane control model is adaptive.

13. The medium of claim 8 , wherein the selected driving lane control model characterizes driving lane control behavior of the plurality of people when in the current driving condition.

14. The medium of claim 8 , wherein the set of visual images is a first set of visual images, the medium further storing instructions to cause the machine to:

analyze a second set of visual images derived from the real time sensor data, to detect a presence of a passenger within the autonomous vehicle; and

determine, in response to detecting the presence of the passenger, a membership of the passenger in the plurality of people.

15. A system, comprising:

a driving lane detector implemented by a processor and configured to:

receive a set of visual images derived from real time sensor data and representing a surface of a road on which an autonomous vehicle is positioned, and

detect, on-the-fly and based on the set of visual images and at least one machine learning based lane detection model, marks on the road, the marks representing a lane of the road in which the autonomous vehicle is positioned;

a driving lane planning unit implemented by the processor and configured to:

receive a plurality of driving lane control models, each driving lane control model from the plurality of driving lane control models derived based on recorded human driving data associated with a plurality of people under a plurality of different driving conditions,

select a driving lane control model from the plurality of driving lane control models based on a current driving condition associated with a current location of the autonomous vehicle, the current driving condition determined based on self-aware capability parameters of the autonomous vehicle, the self-aware capability parameters at least one of indicative or predictive of an operational capability of the autonomous vehicle when in the current location, and

plan lane control for the autonomous vehicle, via the selected driving lane control model, to generate a lane plan for human-like lane control behavior under the current driving condition, based on the lane in which the autonomous vehicle is positioned and the self-aware capability parameters.

16. The system of claim 15 , wherein the lane plan includes at least one of a lane following plan or a lane changing plan.

17. The system of claim 15 , wherein the self-aware capability parameters include intrinsic capability parameters and extrinsic capability parameters, the intrinsic capability parameters specifying conditions internal to the autonomous vehicle that limit the operational capability of the autonomous vehicle, and the extrinsic capability parameters specifying conditions external to the autonomous vehicle that impact the operational capability of the autonomous vehicle.

18. The system of claim 15 , further comprising an updater configured to dynamically update the self-aware capability parameters to reflect a scenario currently associated with the autonomous vehicle.

19. The system of claim 15 , further comprising an updater configured to dynamically update the recorded human driving data such that the lane control model is adaptive.

20. The system of claim 15 , wherein the selected driving lane control model characterizes driving lane control behavior of the plurality of people when in the current driving condition.

21. The system of claim 15 , wherein the set of visual images is a first set of visual images and the driving lane planning unit is further configured to:

analyze a second set of visual images derived from the real time sensor data, to detect a presence of a passenger within the autonomous vehicle; and

determine, in response to detecting the presence of the passenger, a membership of the passenger in the plurality of people.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 19, 2021
From: PLUSAI LIMITED
To: PLUSAI, INC.
Reel/Frame 056909/0574 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 2, 2021
From: PLUSAI CORP
To: PLUSAI LIMITED
Reel/Frame 055461/0122 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2017
From: ZHENG, HAO; LIU, DAVID WANQIAN; DALY, JR., TIMOTHY PATRICK
To: PLUSAI CORP
Reel/Frame 044422/0673 →
Continuity (1)
Related Publication 20190185011A1 · Jun 20, 2019
Cited By (1)
US 12,307,347