IP Library Granted Patent US 11,392,131
Granted Patent B2
US 11,392,131 · App. 16/287,937 · Granted Jul 19, 2022

Method for determining driving policy

Inventors: Ravi Kumar Satzoda (Palo Alto, CA); Suchitra Sathyanarayana (Palo Alto, CA); Ludmila Levkova (Palo Alto, CA); Stefan Heck (Palo Alto, CA)
Assignee: Nauto, Inc.
G05D1/0221B60W40/09G05D1/0246G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,392,131
App. No.
16/287,937
Filed
Feb 27, 2019
Granted
Jul 19, 2022
Kind
B2
Examiner
TO, TUAN C
Art Unit
3667
USPC
701/23
Abstract

Systems and methods for driving. A driving data set for each of plurality of human-driven vehicles is determined. For each driving data set, exterior scene features of an exterior scene of the respective vehicle are extracted from the exterior image data. A driving response model is trained based on the exterior scene features and the vehicle control inputs from the selected driving data sets.

Claims (42)

1. A method, comprising:

obtaining driving data sets for respective vehicles, each of the driving data sets comprising synchronized interior image data and exterior image data, wherein the synchronized interior image data and exterior image data are associated with vehicle control inputs;

for each of the driving data sets, determining a driving quality metric that indicates a driving quality;

selecting one of the driving data sets, having the driving quality metric satisfying a condition; and

providing the exterior image data from the selected one of the driving data sets for training a driving response model based on the exterior image data from the selected one of the driving data sets.

2. The method of claim 1 , further comprising:

for each of the driving data sets:

determining driver attention based on the interior image data of the corresponding driving data set; and

determining a region of interest (ROI) based on driver attention.

3. The method of claim 1 , wherein the driving response model is trained based on an exterior scene feature extracted from the exterior image data of the selected one of the driving data sets.

4. The method of claim 1 , wherein one of the driving quality metrics is determined based on the interior image data of the corresponding driving data set.

5. The method of claim 1 , wherein one of the driving quality metrics comprises an attentiveness score, determined based on driver gaze, wherein the driver gaze is extracted from the interior image data of the corresponding driving data set.

6. The method of claim 5 , wherein the attentiveness score is determined based on the driver gaze and a region of interest identified in the exterior image data of the corresponding driving data set.

7. The method of claim 1 , wherein one of the driving quality metrics comprises a driver score.

8. The method of claim 1 , wherein one of the driving quality metrics is determined based on a comparison of driver behavior, indicated by the corresponding driving data set, with an expected driving behavior.

9. The method of claim 1 , wherein one of the driving quality metrics comprises a ride comfort metric.

10. The method of claim 1 , wherein at least one of the driving data sets is obtained by:

detecting a predetermined driving event; and

recording the synchronized interior image data and exterior image data for the at least one of the driving data sets after detecting the predetermined driving event.

11. The method of claim 1 , wherein the synchronized interior image data and exterior image data of the driving data sets are obtained by a remote computing system, and wherein the act of determining the driving quality metrics, the act of selecting the one of the driving data sets, and the act of providing the exterior image data from the selected one of the driving data sets for training the driving response model, are performed by the remote computing system.

12. The method of claim 1 , wherein the driving response model is trained for different driving events using different respective ones of the driving data sets.

13. The method of claim 1 , further comprising extracting exterior scene feature from the exterior image data of one of the driving data sets by:

determining a point cloud based on the exterior image data of the one of the driving data sets; and

extracting the exterior scene feature from the point cloud.

14. The method of claim 1 , wherein the act of obtaining driving data sets and the act of determining driving quality metrics for the respective driving data sets are performed by a vehicle onboard system.

15. The method of claim 1 , wherein the driving response model is trained based on the exterior image data of the selected one of the driving data sets, and also based on the vehicle control inputs associated with the selected one of the driving data sets.

16. The method of claim 1 , wherein the driving response model is trained based on the exterior image data of the selected one of the driving data sets, and also based on driver behavior that is determined based on the interior image data of the selected one of the driving data sets.

17. A system, comprising:

an interior camera configured to generate interior image data of a vehicle;

an exterior camera configured to generate exterior image data of the vehicle; and

a processing system configured to:

obtain a driving data set for a driving session that includes the interior image data and the exterior image data provided by the interior camera and the exterior camera, respectively, wherein the interior image data and the exterior image data are synchronized with each other and are associated with vehicle control inputs;

determine a driving quality metric for the driving data set, the driving quality metric indicating a driving quality; and

providing the driving data set for training a driving response model, wherein the driving response model is based on an external scene feature extracted from the exterior image data and a driver behavior extracted from the interior image data.

18. The system of claim 17 , further comprising:

a scanning model configured to determine a region of interest (ROI) for the exterior image data; and

a gaze detector configured to extract an eye gaze of a driver from the interior image data;

wherein the system is configured to use the ROI and the eye gaze to determine the driving quality metric for the driving data set.

19. The system of claim 17 , further comprising:

the driving response model,

wherein the system is configured to generate a notification when a driving behavior differs from an expected driving behavior.

20. The system of claim 17 , wherein the driving response model is also based on the vehicle control inputs.

Assignments (4)
RELEASE OF SECURITY INTEREST Recorded Jun 19, 2026
From: ORIX GROWTH CAPITAL, LLC
To: NAUTO, INC.
Reel/Frame 075016/0824 →
SECURITY INTEREST Recorded Aug 8, 2025
From: NAUTO, INC.
To: ORIX GROWTH CAPITAL, LLC, AS AGENT
Reel/Frame 071976/0818 →
SECURITY INTEREST Recorded Nov 10, 2022
From: NAUTO, INC.
To: SILICON VALLEY BANK
Reel/Frame 061722/0392 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 9, 2019
From: SATZODA, RAVI KUMAR; SATHYANARAYANA, SUCHITRA; LEVKOVA, LUDMILA; HECK, STEFAN
To: NAUTO, INC.
Reel/Frame 048834/0624 →
Continuity (3)
Provisional Application 62729350 · Sep 10, 2018
Provisional Application 62635701 · Feb 27, 2018
Related Publication 20190265712A1 · Aug 29, 2019