IP Library Patent Application 16019828
Patent Application
App. No. 16/019,828

SYSTEM AND METHOD FOR IMAGE ANALYSIS

Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US None
App. No.
16/019,828
Abstract

A method for image analysis, including recording an image sequence at a vehicle system mounted to a vehicle; automatically detecting an object within the image sequence with a detection module; automatically defining a bounding box about the detected object within each image of the image sequence; modifying the image sequence with the bounding boxes for the detected object to generate a modified image sequence; at a verification module associated with the detection module, labeling the modified image sequence as comprising one of a false positive, a false negative, a true positive, and a true negative detected object based on the bounding box within at least one image of the modified image sequence; training the detection module with the label for the modified image sequence; and automatically detecting objects within a second image sequence recorded with the vehicle system with the trained detection module.

Claims (59)

1 . A method for image analysis, comprising:

detecting expert driver behavior at a motion sensor of a vehicle system mounted to a vehicle;

in response to detecting expert driver behavior, recording an image at the vehicle system;

automatically detecting an object within the image with a first module;

automatically defining a first bounding box about the detected object;

modifying the image with the first bounding box for the detected object to generate a first modified image;

at a first verification module associated with the first module, labeling the first bounding box within the modified image with a first label as one of a false positive, a false negative, a true positive, and a true negative detected object;

training the first module with the first label for the first bounding box; and

automatically detecting objects within a second image, recorded at the vehicle system, with the trained first module.

2 . The method of claim 1 , further comprising automatically labeling the first bounding box with a second label corresponding to the expert driving behavior, and training the first module with the first label and the second label.

3 . The method of claim 1 , wherein the motion sensor comprises an accelerometer that outputs an accelerometer signal indicative of vehicle motion, and wherein detecting expert driver behavior comprises recording the accelerometer signal exceeding a threshold amplitude.

4 . The method of claim 1 , wherein the first module is executed by the vehicle system, the method further comprising transmitting the first modified image from the vehicle system to a remote computing system, remote from the vehicle, based on the object detected within the image.

5 . The method of claim 1 , further comprising:

automatically assigning an object class to the detected object with a second module to generate a classified object;

modifying the image with a second bounding box for the classified object to generate a second modified image, wherein the second modified image comprises a set of pixels;

at a second verification module associated with the object class, labeling the second bounding box within the second modified image with a second label as one of a false positive, a false negative, a true positive, and a true negative for the object class;

automatically labeling the second bounding box with a third label corresponding to the expert driving behavior; and

training the second module with the second label and the third label for the second bounding box within the second modified image;

6 . The method of claim 5 , wherein the object class is one of a plurality of object classes, and wherein the second verification module is one of a plurality of verification modules, each associated with a different object class of the plurality of object classes.

7 . The method of claim 1 , wherein the first and second module are a first and second level of a cascaded classification system.

8 . The method of claim 1 , wherein the second modified image defines a set of pixels, and wherein labeling the second bounding box comprises:

automatically determining a horizon line in the second modified image based on image fiducials, and

automatically labeling the detected object based on a relative location in the second modified image between the horizon line and the detected object.

9 . The method of claim 1 , further comprising:

determining a score for the driver based on historic driving sessions associated with the driver;

labeling the bounding box with a second label based on the score; and

training the first module with the first label and second label for the bounding box when the driver has score above a predetermined threshold.

10 . The method of claim 1 , wherein the first and second verification modules each comprise a user interface, wherein labeling comprises:

receiving a user selection of the bounding box within the modified image, wherein receiving the user selection automatically labels the bounding box as a false positive.

11 . The method of claim 1 , further comprising receiving a user boundary input about an image region, generating a user bounding box about the image region based on the user boundary input, and wherein receiving the user boundary input automatically labels the user bounding box as a false negative.

12 . The method of claim 1 , further comprising:

recording a sequential image, wherein the image is a first frame of an image sequence recorded with the vehicle system, and the sequential image is a second frame of the image sequence;

automatically defining a tracked bounding box about the detected object within the second frame based upon a predicted trajectory of the detected object between the first and second frame; wherein the predicted trajectory is determined by a tracking module;

modifying the second frame with the tracked bounding box to generated a modified second frame;

at the verification module, labeling the tracked bounding box within the modified second frame as one of a false positive, a false negative, a true positive, and a true negative based on a comparison between the predicted trajectory and an actual trajectory of the detected object between the first and second frame;

training the tracking module with the label for the tracked bounding box; and

automatically tracking detected objects within a second image sequence recorded with the vehicle system based on the trained tracking module.

13 . A method for image analysis, comprising:

recording an image sequence at a vehicle system mounted to a vehicle;

automatically detecting an object within the image sequence with a detection module;

automatically defining a bounding box about the detected object within each image of the image sequence;

modifying the image sequence with the bounding boxes for the detected object to generate a modified image sequence;

at a verification module associated with the detection module, labeling the modified image sequence with a first label comprising one of a false positive, a false negative, a true positive, and a true negative detected object based on the bounding box within at least one image of the modified image sequence;

determining driver behavior of a driver operating the vehicle using a motion sensor of the vehicle system;

labeling the modified image sequence with a second label comprising the driver behavior;

training the detection module with the first label and the second label for the modified image sequence; and

automatically detecting objects within a second image sequence recorded with the vehicle system with the trained detection module.

14 . The method of claim 13 , further comprising:

automatically assigning an object class to the detected object with a classification module to generate a classified object;

modifying the image sequence with a second bounding box for the classified object within each image of the image sequence to generate a second modified image sequence;

at a second verification module associated with the object class, labeling the second modified image sequence with a third label comprising one of a false positive, a false negative, a true positive, and a true negative for the object class based on the second bounding box within at least one image of the second modified image sequence; and

training the classification module with the third label for the second bounding box within the second modified image sequence and the second label; and

automatically classifying objects within the second image sequence recorded with the vehicle system with the trained classification module.

15 . The method of claim 14 , wherein the object class is determined based on a roadway type on which the vehicle is driving, and wherein the roadway type on which the vehicle is driving is determined by a vibration sensor of the vehicle system.

16 . The method of claim 13 , further comprising determining a score for the driver, based on historic driving sessions associated with the driver.

17 . The method of claim 16 , wherein the detection module is trained with the first label and second label for the modified image sequence when the driver has score above a predetermined threshold, wherein the second label is determined based on the score.

18 . The method of claim 13 , wherein the motion sensor comprises an accelerometer that outputs an accelerometer signal indicative of vehicle motion, and wherein determining driver behavior comprises recording the accelerometer signal exceeding a threshold amplitude

19 . The method of claim 18 , wherein recording the image sequence is performed in response to the accelerometer signal exceeding the threshold amplitude.

20 . The method of claim 13 , wherein automatically defining the bounding box about the detected object comprises defining a bounding box within a first image of the image sequence, and further comprising defining bounding boxes around the object within subsequent images of the image sequence based on a predicted trajectory of the object, wherein the predicted trajectory is based on the object class.

Assignments (5)
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE STREET ADDRESS PREVIOUSLY RECORDED AT REEL: 047977 FRAME: 0299. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jun 14, 2019
From: NAUTO GLOBAL LIMITED
To: NAUTO, INC.
Reel/Frame 049475/0814 →
CORRECTIVE ASSIGNMENT TO CORRECT THE STATE OF INCORPORATION INSIDE THE ASSIGNMENT DOCUMENT PREVIOUSLY RECORDED AT REEL: 047821 FRAME: 0958. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 21, 2018
From: NAUTO GLOBAL LIMITED
To: NAUTO, INC.
Reel/Frame 047977/0299 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 19, 2018
From: NAUTO GLOBAL INC.
To: NAUTO, INC.
Reel/Frame 047821/0958 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 27, 2018
From: SATZODA, RAVI KUMAR; SATHYANARAYANA, SUCHITRA
To: NAUTO, INC.
Reel/Frame 046212/0685 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 27, 2018
From: NAUTO, INC.
To: NAUTO GLOBAL LIMITED
Reel/Frame 046212/0713 →