IP Library Granted Patent US 11,113,961
Granted Patent B2
US 11,113,961 · App. 16/704,520 · Granted Sep 7, 2021

Driver behavior monitoring

Inventors: David Jonathan Julian (San Diego, CA); Avneesh Agrawal (Bangaluru, IN)
Assignee: NETRADYNE, INC.
G08G1/04G06K9/00791G06K9/00845G06K9/6271G07C5/0808G07C5/0866G08G1/0112G08G1/0133B60W40/09B60W2420/42
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,113,961
App. No.
16/704,520
Granted
Sep 7, 2021
Kind
B2
Abstract

Systems and methods provide, implement, and use using a computer-vision based methods of context-sensitive monitoring and characterization of driver behavior. Additional systems and methods are provided for unsupervised learning of action values, monitoring of a driver's environment, and transmitting visual information from a client to a server.

Claims (50)

1. A method comprising:

detecting, by at least one processor of a computing device in communication with a camera, the presence of a traffic light, a state of the traffic light, and an intersection marking in a field of view of the camera, wherein the camera is mounted on or in a vehicle; and

determining, by the at least one processor, that the vehicle passed the intersection marking after the traffic light turned red, based at least in part on first visual data captured by the camera,

wherein detecting the presence of the traffic light further comprises:

initially receiving, by the at least one processor, the first visual data; and

identifying, by the at least one processor, a location of the traffic light in the first visual data, and

wherein determining that the vehicle passed the intersection marking after the traffic light turned red further comprises:

identifying, by the at least one processor, that the traffic light is in a red state in the first visual data;

determining, by the at least one processor, that the vehicle was not in the intersection at a first time corresponding to when the visual data was captured;

receiving, by the at least one processor, second visual data captured by the camera, wherein the second visual data is captured by the camera subsequent to the capture of the first visual data;

identifying, by the at least one processor, a second location of the traffic light in the second visual data; based at least in part on the first location of the traffic light in the first visual data;

identifying, by the at least one processor, that the traffic light is in a red state in the second visual data; and

determining, by the at least one processor, that the vehicle was in the intersection at a second time; wherein the second time corresponds to when the second visual data was captured.

2. The method of claim 1 , wherein determining that the vehicle passed the intersection marking after the traffic light turned red further comprises:

determining, by the at least one processor, a distance between the vehicle and the intersection at the first time, wherein the traffic light is associated with the intersection; and

determining, by the at least one processor, that the vehicle did not stop before passing through the intersection.

3. The method of claim 1 , wherein determining that the vehicle passed the intersection marking after the traffic light turned red further comprises:

recognizing, by the at least one processor, an intersection marking in the visual data; and

determining, by the at least one processor, that the vehicle did not stop before passing through the intersection.

4. The method of claim 1 , wherein identifying the location of the traffic light comprises:

determining, by the at least one processor, a bounding box for the traffic light; and

determining, by the at least one processor, a probability that a portion of the visual data associated with the bounding box contains a traffic light.

5. The method of claim 1 , wherein the computing device in communication with the camera is located in or on the vehicle.

6. The method of claim 1 , further comprising:

determining, by the at least one processor, that the vehicle did not stop before passing through the intersection based on the visual data captured by the camera alone.

7. A method comprising:

detecting, by at least one processor of a computing device in communication with a camera, at a first time, the presence of a traffic light, a state of the traffic light, and an intersection marking in a field of view of the camera, wherein the camera is mounted on or in a vehicle;

detecting, by the at least one processor, that the traffic light turned yellow at a second time subsequent to the first time;

determining, by the at least one processor, a distance between the vehicle and the intersection marking at the second time, wherein the traffic light is associated with the intersection marking; and

determining, by the at least one processor, a speed of the vehicle at the second time;

determining, by the at least one processor, that the vehicle did not stop at the intersection;

determining, by the at least one processor, that the traffic light turned red before the vehicle was in the intersection; and

determining, by the at least one processor, that the vehicle appropriately ran a red light based on at least one of: the determined distance between the vehicle and the intersection at the first time, the speed of the vehicle at the second time, or some combination thereof.

8. The method of claim 7 , wherein detecting the presence of a traffic light further comprises:

initially receiving, by the at least one processor, first visual data captured by the camera at the first time;

identifying, by the at least one processor, a first location of the traffic light in the first visual data; and

determining, by the at least one processor, that the traffic light is in a green state in the first visual data;

and wherein detecting that the traffic light turned yellow further comprises:

receiving, by the at least one processor, second visual data captured by the camera, wherein the second visual data is captured by the camera at the second time;

identifying, by the at least one processor, a second location of the traffic light in the second visual data; based at least in part on the first location of the traffic light in the first visual data; and

identifying, by the at least one processor, that the traffic light is in a yellow state in the second visual data.

9. A method of claim 7 , further comprising:

detecting, by at least one processor of a computing device in communication with a camera, at a first time, a presence of a traffic light, wherein the camera is mounted on or in a vehicle;

determining, by the at least one processor, that the vehicle stopped at the traffic light;

determining, by the at least one processor, that the traffic light turned green;

measuring, by the at least one processor, an elapsed time between the light turning green and the vehicle starting forward;

assessing, by the at least one processor, that a driver of the vehicle is being responsive to traffic signals based at least in part on the elapsed time;

detecting, by the at least one processor, a pedestrian based on visual data captured by the camera; and

grading, by the at least one processor, the driver positively for waiting at a green light based on the detected pedestrian.

10. The method of claim 9 , further comprising determining an extent to which a driver obeys traffic lights, based at least in part on the determination that the vehicle stopped at the traffic light.

Assignments (4)
SECURITY INTEREST Recorded Apr 6, 2026
From: NETRADYNE, INC.
To: FIRST-CITIZENS BANK & TRUST COMPANY
Reel/Frame 075435/0670 →
SECURITY INTEREST Recorded Apr 6, 2026
From: NETRADYNE, INC.
To: FIRST-CITIZENS BANK & TRUST COMPANY
Reel/Frame 075359/0194 →
SECURITY INTEREST Recorded Mar 9, 2021
From: NETRADYNE, INC.
To: SILICON VALLEY BANK
Reel/Frame 055530/0593 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 5, 2019
From: JULIAN, DAVID JONATHAN; AGRAWAL, AVNEESH
To: NETRADYNE, INC.
Reel/Frame 051192/0750 →
Continuity (5)
Continuation 16507894 · Jul 10, 2019
Continuation 15437646 · Feb 21, 2017
Continuation PCTUS2017013062 · Jan 11, 2017
Provisional Application 62277470 · Jan 11, 2016
Related Publication 20200111350A1 · Apr 9, 2020
Cited By (1)
US 12,187,306