IP Library › Granted Patent US 9,686,451
Granted Patent B2
US 9,686,451 · App. 14/602,214 · Granted Jun 20, 2017

Real time driving difficulty categorization

Inventors: Preeti Jayagopi Pillai (Santa Clara, CA); Veeraganesh Yalla (Sunnyvale, CA); Kentaro Oguchi (Menlo Park, CA); Hirokazu Nomoto (Aichi-ken, JP)
H04N5/225G06K9/00791G06K9/46G06T2207/10016
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,686,451
App. No.
14/602,214
Granted
Jun 20, 2017
Kind
B2
Abstract

The disclosure includes a system and method for determine a real time driving difficulty category. The method may include determining image feature vector data based on one or more features depicted in a real time image of a road scene. The image feature vector data may describe an image feature vector for an edited version of the real time image. The method may include determining offline road map data for the road scene, which includes a static label for a road included in the road scene and offline road information describing a regulatory speed limit for the road. The method may include selecting, based on the static label, a classifier for analyzing the image feature vector. The method may include executing the selected classifier to determine a real time driving difficulty category describing the difficulty for a user of the client device to drive in the road scene.

Claims (36)

1. A method comprising:

receiving real time image data describing a real time image depicting a road scene;

determining region of interest data describing a version of the real time image configured to remove redundant features from the real time image;

determining dense grid sampling data describing one or more features included in the version of the real time image described by the region of interest data;

determining image feature vector data based on the one or more features included in the dense grid sampling data, the image feature vector data describing an image feature vector for the version of the real time image described by the region of interest data;

receiving client location data describing a geographic location of a client device associated with the road scene included in the real time image;

determining offline road map data for the road scene based on the geographic location of the client device, the offline road map data describing (1) a static label for a road included in the road scene and (2) offline road information describing a regulatory speed limit for the road included in the road scene;

selecting, based on the static label, a classifier for analyzing the image feature vector; and

executing, by a processor, the selected classifier to determine a real time driving difficulty category describing the difficulty for a user of the client device to drive in the road scene depicted in the real time image, wherein the real time driving difficulty category is determined based on the image feature vector, offline road information and runtime road description data describing the real time driving conditions of the road scene depicted in the real time image.

2. A method comprising:

determining dense grid sampling data describing one or more features included in a version of a real time image of a road scene which is configured to remove one or more redundant features from the real time image;

determining image feature vector data based on one or more features described by the dense grid sampling data, the image feature vector data describing an image feature vector for the version of the real time image which is configured to remove the one or more redundant features from the real time image;

determining offline road map data for the road scene based on a geographic location of a client device associated with the road scene included in the real time image, the offline road map data describing (1) a static label for a road included in the road scene and (2) offline road information describing a regulatory speed limit for the road included in the road scene;

selecting, based on the static label, a classifier for analyzing the image feature vector; and

executing, by a processor, the selected classifier to determine a real time driving difficulty category describing the difficulty for a user of the client device to drive in the road scene depicted in the real time image, wherein the real time driving difficulty category is determined based on the image feature vector and the offline road information.

3. The method of claim 2 , wherein the classifier is executed in real time relative to when the real time image depicting the road scene is captured.

4. The method of claim 2 , wherein the real time driving difficulty category is determined based in part on runtime road description data describing the real time driving conditions of the road scene depicted in the real time image.

5. The method of claim 2 , wherein the client device is a mobile device.

6. The method of claim 2 , wherein the real time image is captured by an onboard image system.

7. The method of claim 6 , wherein the client device is a vehicle and the onboard image system is mounted on the vehicle to be frontward facing.

8. The method of claim 6 , wherein the client device is a vehicle and the onboard image system is mounted on the vehicle to be rearward facing.

9. The method of claim 2 , wherein the offline road map data is determined during an offline process.

10. The method of claim 2 , wherein the real time image is captured during a runtime process.

11. A non-transitory computer-readable medium having computer instructions stored thereon that are executable by a processing device to perform or control performance of operations comprising:

determining dense grid sampling data describing one or more features included in a version of a real time image of a road scene which is configured to remove one or more redundant features from the real time image;

determining image feature vector data based on one or more features described by the dense grid sampling data, the image feature vector data describing an image feature vector for flail the version of the real time image which is configured to remove the one or more redundant features from the real time image;

determining offline road map data for the road scene based on a geographic location of a client device associated with the road scene included in the real time image, the offline road map data describing (1) a static label for a road included in the road scene and (2) offline road information describing a regulatory speed limit for the road included in the road scene;

selecting, based on the static label, a classifier analyzing the image feature vector; and

executing, by a processor, the selected classifier to determine a real time driving difficulty category describing the difficulty for a user of the client device to drive in the road scene depicted in the real time image, wherein the real time driving difficulty category is determined based on the image feature vector and the offline road information.

12. The non-transitory computer-readable medium of claim 11 , wherein the classifier is executed in real time relative to when the real time image depicting the road scene is captured.

13. The non-transitory computer-readable medium of claim 11 , wherein the client device is a robot.

14. The non-transitory computer-readable medium of claim 11 , wherein the client device is an IoT device.

15. The non-transitory computer-readable medium of claim 11 , wherein the real time image is captured by an onboard image system.

16. The non-transitory computer-readable medium of claim 15 , wherein the client device is a vehicle and the onboard image system is mounted on the vehicle to be frontward facing.

17. The non-transitory computer-readable medium of claim 15 , wherein the client device is a vehicle and the onboard image system is mounted on the vehicle to be rearward facing.

18. The non-transitory computer-readable medium of claim 15 or 17 , wherein: the offline road map data is determined during an offline process; and the real time image is captured during a runtime process.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 21, 2015
From: PILLAI, PREETI JAYAGOPI; YALLA, VEERAGANESH; OGUCHI, KENTARO; NOMOTO, HIROKAZU
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 034780/0400 →
Continuity (1)
Related Publication 20160207458A1 · Jul 21, 2016