IP Library Granted Patent US 11,069,257
Granted Patent B2
US 11,069,257 · App. 14/540,825 · Granted Jul 20, 2021

System and method for detecting a vehicle event and generating review criteria

Inventors: Jason Palmer (Carlsbad, CA); Slaven Sljivar (San Diego, CA); Mark Freitas (San Diego, CA); Daniel A. Deninger (Carlsbad, CA); Jeffrey Todd Griswold (San Diego, CA)
Assignee: SmartDrive Systems, Inc.
G09B19/167B60R1/00G06K9/00711G06K9/00791G09B5/02B60R2300/105B60R2300/205B60R2300/80G06K2009/00738
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,069,257
App. No.
14/540,825
Granted
Jul 20, 2021
Kind
B2
Abstract

This disclosure relates to a system and method for detecting vehicle events and generating review criteria based on the detected vehicle events. Some or all of the system may be installed in a vehicle and/or be otherwise coupled with a vehicle. The system may include one or more sensors configured to generate output signals conveying information related to the vehicle and/or multiple video capture devices configured to acquire visual output information representing a vehicle environment. In some implementations, the system may determine a vehicle event type based on the information conveyed by the output signals. The system may generate review criteria, which correspond to the vehicle event, based on the vehicle event type and the fields of view corresponding to the video capture devices.

Claims (86)

1. A system configured to detect a vehicle event and generate review criteria for manual review, wherein the review criteria correspond to the vehicle event, a portion of the system configured to couple with a vehicle, the system comprising:

multiple video capture devices configured to acquire visual output information representing a vehicle environment, the individual video capture devices having separate corresponding fields of view of the vehicle environment, the vehicle environment including spaces in and around an interior and an exterior of the vehicle;

one or more sensors configured to generate output signals conveying vehicle operation information related to the operation and/or context of the vehicle; and

one or more physical computer processors configured by computer readable instructions to:

detect, in real-time or near real-time, the vehicle event based on the vehicle operation information conveyed by the output signals;

determine a vehicle event type based on the vehicle operation information conveyed by the output signals;

responsive to the detection of the vehicle event, generate one or more review criteria that correspond to the detected vehicle event, wherein the one or more review criteria are generated based on the vehicle event type, wherein the one or more review criteria include one or more of a question, an inquiry, or a prompt for a manual reviewer, wherein the one or more review criteria are used to prompt the manual reviewer to provide manual reviewer input describing the detected vehicle event based on the acquired visual output information;

facilitate communication of:

(i) the vehicle operation information,

(ii) the acquired visual output information, and

(iii) the generated one or more review criteria to a remote computing device that is remote and separate from the vehicle;

facilitate, at the remote computing device, display of a customizable reviewer interface, wherein the customizable reviewer interface displays, to the manual reviewer:

(i) the generated one or more review criteria to prompt the manual reviewer to provide the manual reviewer input describing the detected vehicle event based on the acquired visual output information,

(ii) a portion of the vehicle operation information generated during the detected vehicle event,

(iii) an input field configured to accept the manual reviewer input from the manual reviewer describing the detected vehicle event, and

(iv) a portion of the acquired visual output information acquired during the detected vehicle event, wherein the portion of the acquired visual output information includes individual ones of the fields of view corresponding to the video capture devices, wherein the manual reviewer input received from the manual reviewer is responsive to one or more of the question, the inquiry, or the prompt describing the detected vehicle event, and wherein the manual reviewer input is based on the acquired visual output information; and

generate a vehicle event record for the vehicle event, wherein the vehicle event record includes the manual reviewer input.

2. The system of claim 1 , wherein the generated one or more review criteria are further generated based on corresponding fields of view of individual non-system integrated video capture devices.

3. The system of claim 1 , wherein the detection of the vehicle event is based on the vehicle operation information conveyed by the output signals that correspond to a set of parameters that describe the vehicle event.

4. The system of claim 1 , wherein the vehicle event type is further determined based on the acquired visual output information acquired from the multiple video capture devices.

5. The system of claim 1 , wherein the one or more physical computer processors are further configured, responsive to a review of the detected vehicle event and/or previous detected vehicle events of the same vehicle event type by the manual reviewer, to generate coaching session information that corresponds to the detected vehicle event and/or previous detected vehicle events for use in a coaching session, wherein the coaching session information is different from the manual reviewer input.

6. The system of claim 1 , wherein the one or more physical computer processors are further configured, responsive to the detection of the vehicle event, to facilitate electronic storage of the of the vehicle operation information and the acquired visual output information.

7. The system of claim 1 , wherein the one or more physical computer processors are further configured to generate a performance score for a vehicle operator of the vehicle at the time of the detected vehicle event based on the manual review by the manual reviewer.

8. The system of claim 1 , wherein the generated one or more review criteria include first review criteria for a first individual field of view corresponding to a first individual video capture device and second review criteria for a second individual field of view corresponding to a second individual video capture device, and wherein the first review criteria is different from the second review criteria based on the first individual field of view being different from the second individual field of view.

9. A system configured to detect a vehicle event and generate review criteria for manual review, wherein the review criteria correspond to the vehicle event, at least a portion of the system configured to couple with a vehicle, the system comprising:

multiple video capture devices configured to acquire visual output information representing a vehicle environment from individual video capture devices having separate corresponding fields of view of the vehicle environment, the vehicle environment including spaces in and around an interior and an exterior of the vehicle, the visual output information including a first visual output information and second visual output information,

the first visual output information having a digital format and being generated by an individual one of the multiple video capture devices that is a system integrated video capture device, and

the second visual output information having an analog format and being generated by an individual one of the multiple video capture devices that is a non-system integrated video capture device;

one or more sensors configured to generate output signals conveying vehicle operation information related to the operation and/or context of the vehicle;

an adapter configured to convert analog information to digital information such that the second visual output information having the analog format is converted into second visual output information having a digital format; and

one or more physical computer processors configured by computer readable instructions to:

detect, in real-time or near real-time, the vehicle event based on the vehicle operation information conveyed by the output signals;

determine a vehicle event type based on the vehicle operation information conveyed by the output signals;

responsive to the detection of the vehicle event, generate one or more review criteria that corresponds to the vehicle event, wherein the one or more review criteria are generated based on the vehicle event type, wherein the one or more review criteria include one or more of a question, an inquiry, or a prompt for a manual reviewer, wherein the one or more review criteria are used to prompt the manual reviewer to provide manual reviewer input describing the detected vehicle event based on the acquired visual output information;

facilitate communication of:

(i) the vehicle operation information,

(ii) the acquired visual output information, and

(iii) the generated one or more review criteria to a remote computing device that is remote and separate from the vehicle; and

facilitate, at the remote computing device, display of a customizable reviewer interface, wherein the customizable reviewer interface displays, to the manual reviewer:

(i) the generated one or more review criteria to prompt the manual reviewer to provide the manual reviewer input describing the detected vehicle event based on the acquired visual output information,

(ii) a portion of the vehicle operation information generated during the detected vehicle event,

(iii) an input field configured to accept the manual reviewer input from the manual reviewer describing the detected vehicle event, and

(iv) a portion of the acquired visual output information acquired during the detected vehicle event, wherein the portion of the acquired visual output information includes individual ones of the fields of view corresponding to the video capture devices, wherein the manual reviewer input received from the manual reviewer is responsive to one or more of the question, the inquiry, or the prompt describing the detected vehicle event, and wherein the manual reviewer input is based on the acquired visual output information.

10. The system of claim 8 , wherein the adapter is further configured to facilitate adapting one or more characteristics of the first visual output information and/or the second visual output information, the characteristics including one or more of a frame rate, a resolution, a brightness, a contrast, a saturation, or an image correction.

11. The system of claim 9 , wherein the generated one or more review criteria include first review criteria for a first individual field of view corresponding to a first individual video capture device and second review criteria for a second individual field of view corresponding to a second individual video capture device, and wherein the first review criteria is different from the second review criteria based on the first individual field of view being different from the second individual field of view.

12. A method for detecting a vehicle event and generating review criteria for manual review, wherein the review criteria correspond to the vehicle event, the method comprising:

acquiring visual output information representing a vehicle environment from individual video capture devices having separate corresponding fields of view of the vehicle environment, the vehicle environment including spaces in and around an interior and an exterior of the vehicle;

generating output signals conveying vehicle operation information related to the operation and/or context of the vehicle;

detecting, in real-time or near real-time, the vehicle event based on the vehicle operation information conveyed by the output signals;

determining a vehicle event type based on the vehicle operation information conveyed by the output signals;

generating one or more review criteria that correspond to the detected vehicle event, wherein the one or more review criteria are generated based on the vehicle event type, wherein the one or more review criteria include one or more of a question, an inquiry, or a prompt for a manual reviewer, wherein the one or more review criteria are used to prompt the manual reviewer to provide manual reviewer input describing the detected vehicle event based on the acquired visual output information;

facilitating communication of:

(i) the vehicle operation information,

(ii) the acquired visual output information, and

(iii) the generated one or more review criteria to a remote computing device that is remote and separate from the vehicle;

facilitating, at the remote computing device, display of a customizable reviewer interface to the manual reviewer, wherein the customizable reviewer interface displays, to the manual reviewer:

(i) the generated one or more review criteria to prompt the manual reviewer to provide the manual reviewer input describing the detected vehicle event based on the acquired visual output information, wherein the manual reviewer input is based on the acquired visual output information,

(ii) an input field to accept the manual reviewer input from the manual reviewer describing the detected vehicle event,

(iii) a portion of the vehicle operation information generated during the detected vehicle event, and

(iv) a portion of the acquired visual output information acquired during the detected vehicle event, wherein the portion of the acquired visual output information includes individual ones of the fields of view corresponding to the video capture devices; and

accepting, by the input field, the manual reviewer input from the manual reviewer, wherein the manual reviewer input is responsive to one or more of the question, the inquiry, or the prompt describing the detected vehicle event, and wherein the manual reviewer input is based on the acquired visual output information.

13. The method of claim 12 , wherein generating the one or more review criteria is further based on corresponding fields of view of individual non-system integrated video capture devices.

14. The method of claim 12 , further comprising determining a set of parameters that describe the vehicle event for detecting the vehicle event.

15. The method of claim 12 , wherein determining the vehicle event type is further based on the acquired visual output information acquired from the multiple video capture devices.

16. The method of claim 12 , further comprising, responsive to a review of the detected vehicle event and/or previous detected vehicle events of the same vehicle event type by the manual reviewer, generating coaching session information that corresponds to the detected vehicle event and/or previous detected vehicle events for use in a coaching session, wherein the coaching session information is different from the manual reviewer input.

17. The method of claim 12 , further comprising, responsive to the detection of the vehicle event, facilitating electronic storage of the vehicle operation information and the acquired visual output information.

18. The method of claim 12 , further comprising, generating a performance score for a vehicle operator of the vehicle at the time of the vehicle event based on a manual review by the manual reviewer.

19. The method of claim 12 , wherein the generated one or more review criteria include first review criteria for a first individual field of view corresponding to a first individual video capture device and second review criteria for a second individual field of view corresponding to a second individual video capture device, and wherein the first review criteria is different from the second review criteria based on the first individual field of view being different from the second individual field of view.

20. A method for detecting a vehicle event and generating review criteria for manual review, wherein the review criteria correspond to the vehicle event, the method comprising:

acquiring visual output information representing a vehicle environment from individual video capture devices having separate corresponding fields of view of the vehicle environment, the vehicle environment including spaces in and around an interior and an exterior of the vehicle, the visual output information including first visual output information and second visual output information,

the first visual output information having a digital format and being generated by an individual one of the multiple video capture devices that is a system integrated video capture device, and

the second visual output information having an analog format and being generated by an individual one of the multiple video capture devices that is a non-system integrated video capture device;

generating output signals conveying vehicle operation information related to the operation and/or context of the vehicle;

adapting the second visual output information having the analog format into second visual output information having a digital format;

detecting, in real-time or near real-time, the vehicle event based on the vehicle operation information conveyed by the output signals;

determining a vehicle event type based on the vehicle operation information conveyed by the output signals;

responsive to the detection of the vehicle event, generating one or more review criteria that correspond to the detected vehicle event, wherein the one or more review criteria are generated based on the vehicle event type, wherein the one or more review criteria include one or more of a question, an inquiry, or a prompt for a manual reviewer, wherein the one or more review criteria are used to prompt the manual reviewer to provide manual reviewer input describing the detected vehicle event based on the acquired visual output information;

facilitating communication of the vehicle operation information, the acquired visual output information and the generated one or more review criteria to a remote computing device that is remote and separate from the vehicle;

facilitating, at the remote computing device, display of a customizable reviewer interface to the manual reviewer, wherein the customizable reviewer interface displays:

(i) the generated one or more review criteria to prompt the manual reviewer to provide the manual reviewer input describing the detected vehicle event based on the acquired visual output information, wherein the manual reviewer input is based on the acquired visual output information,

(ii) a portion of the vehicle operation information generated during the detected vehicle event,

(iii) an input field to accept the manual reviewer input from the manual reviewer describing the detected vehicle event, and

(iv) a portion of the acquired visual output information acquired during the detected vehicle event, wherein the portion of the acquired visual output information includes individual ones of the fields of view corresponding to the video capture devices; and

accepting, by the input field, the manual reviewer input from the manual reviewer, wherein the manual reviewer input is responsive to one or more of the question, the inquiry, or the prompt describing the detected vehicle event, and wherein the manual reviewer input is based on the acquired visual output information.

21. The method of claim 20 , further comprising adapting one or more characteristics of the first visual output information and/or the second visual output information, the characteristics including one or more of a frame rate, a resolution, a brightness, a contrast, a saturation, or an image correction.

22. The method of claim 20 , wherein the generated one or more review criteria include first review criteria for a first individual field of view corresponding to a first individual video capture device and second review criteria for a second individual field of view corresponding to a second individual video capture device, and wherein the first review criteria is different from the second review criteria based on the first individual field of view being different from the second individual field of view.

Assignments (16)
CORRECTIVE ASSIGNMENT TO CORRECT THE INCORRECT PATENT NUMBER D856640 PREVIOUSLY RECORDED ON REEL 056598 FRAME 0059. ASSIGNOR(S) HEREBY CONFIRMS THE SECOND LIEN PATENT SECURITY AGREEMENT. Recorded Nov 17, 2021
From: OMNITRACS, LLC; ROADNET TECHNOLOGIES, INC.; SMARTDRIVE SYSTEMS, INC.; XRS CORPORATION; HYPERQUEST, LLC (F/K/A HYPERQUEST, INC.); AUDATEX NORTH AMERICA, LLC (F/K/A AUDATEX NORTH AMERICA, INC.); CLAIMS SERVICES GROUP, LLC; DMEAUTOMOTIVE LLC; ENSERVIO, LLC (F/K/A ENSERVIO, INC.); MOBILE PRODUCTIVITY, LLC; SEE PROGRESS, LLC (F/K/A SEE PROGRESS, INC.); SOLERA HOLDINGS, LLC (F/K/A SOLERA HOLDINGS, INC.); EDRIVING FLEET LLC; FINANCE EXPRESS LLC
To: ALTER DOMUS (US) LLC, AS COLLATERAL AGENT
Reel/Frame 058175/0775 →
CORRECTIVE ASSIGNMENT TO CORRECT THE INCORRECT PATENT NUMBER D856640 PREVIOUSLY RECORDED ON REEL 056601 FRAME 0630. ASSIGNOR(S) HEREBY CONFIRMS THE FIRST LIEN PATENT SECURITY AGREEMENT. Recorded Nov 17, 2021
From: OMNITRACS, LLC; ROADNET TECHNOLOGIES, INC.; SMARTDRIVE SYSTEMS, INC.; XRS CORPORATION; HYPERQUEST, LLC (F/K/A HYPERQUEST, INC.); AUDATEX NORTH AMERICA, LLC (F/K/A AUDATEX NORTH AMERICA, INC.); CLAIMS SERVICES GROUP, LLC; DMEAUTOMOTIVE LLC; ENSERVIO, LLC (F/K/A ENSERVIO, INC.); MOBILE PRODUCTIVITY, LLC; SEE PROGRESS, LLC (F/K/A SEE PROGRESS, INC.); SOLERA HOLDINGS, LLC (F/K/A SOLERA HOLDINGS, INC.); EDRIVING FLEET LLC; FINANCE EXPRESS LLC
To: GOLDMAN SACHS LENDING PARTNERS LLC, AS COLLATERAL AGENT
Reel/Frame 058174/0907 →
FIRST LIEN PATENT SECURITY AGREEMENT Recorded Jun 16, 2021
From: OMNITRACS, LLC; ROADNET TECHNOLOGIES, INC.; SMARTDRIVE SYSTEMS, INC.; XRS CORPORATION; HYPERQUEST, LLC (F/K/A HYPERQUEST, INC.); AUDATEX NORTH AMERICA, LLC (F/K/A AUDATEX NORTH AMERICA, INC.); CLAIMS SERVICES GROUP, LLC; DMEAUTOMOTIVE LLC; ENSERVIO, LLC (F/K/A ENSERVIO, INC.); MOBILE PRODUCTIVITY, LLC; SEE PROGRESS, LLC (F/K/A SEE PROGRESS, INC.); SOLERA HOLDINGS, LLC (F/K/A SOLERA HOLDINGS, INC.); EDRIVING FLEET LLC; FINANCE EXPRESS LLC
To: GOLDMAN SACHS LENDING PARTNERS LLC, AS COLLATERAL AGENT
Reel/Frame 056601/0630 →
SECOND LIEN PATENT SECURITY AGREEMENT Recorded Jun 16, 2021
From: OMNITRACS, LLC; ROADNET TECHNOLOGIES, INC.; SMARTDRIVE SYSTEMS, INC.; XRS CORPORATION; HYPERQUEST, LLC (F/K/A HYPERQUEST, INC.); AUDATEX NORTH AMERICA, LLC (F/K/A AUDATEX NORTH AMERICA, INC.); CLAIMS SERVICES GROUP, LLC; DMEAUTOMOTIVE LLC; ENSERVIO, LLC (F/K/A ENSERVIO, INC.); MOBILE PRODUCTIVITY, LLC; SEE PROGRESS, LLC (F/K/A SEE PROGRESS, INC.); SOLERA HOLDINGS, LLC (F/K/A SOLERA HOLDINGS, INC.); EDRIVING FLEET LLC; FINANCE EXPRESS LLC
To: ALTER DOMUS (US) LLC, AS COLLATERAL AGENT
Reel/Frame 056598/0059 →
SECURITY INTEREST RELEASE (REEL/FRAME: 054236/0435) Recorded Jun 8, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS GRANTEE
To: SMARTDRIVE SYSTEMS, INC.
Reel/Frame 056518/0845 →
SECURITY INTEREST RELEASE (REEL/FRAME: 054236/0320) Recorded Jun 8, 2021
From: BARCLAYS BANK PLC, AS GRANTEE
To: SMARTDRIVE SYSTEMS, INC.
Reel/Frame 056520/0944 →
SECURITY INTEREST Recorded Oct 6, 2020
From: SMARTDRIVE SYSTEMS, INC.
To: BARCLAYS BANK PLC, AS COLLATERAL AGENT
Reel/Frame 054236/0320 →
TERMINATION AND RELEASE OF GRANT OF A SECURITY INTEREST - PATENTS Recorded Oct 6, 2020
From: TCS TALENTS, LLC, AS THE COLLATERAL AGENT
To: SMARTDRIVE SYSTEMS, INC.
Reel/Frame 053983/0525 →
SECOND LIEN PATENT SECURITY AGREEMENT Recorded Oct 6, 2020
From: SMARTDRIVE SYSTEMS, INC.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
Reel/Frame 054236/0435 →
RELEASE OF SECURITY INTEREST Recorded Sep 3, 2019
From: ALLY BANK
To: SMARTDRIVE SYSTEMS, INC.
Reel/Frame 050246/0118 →
SECURITY INTEREST Recorded Sep 3, 2019
From: SMARTDRIVE SYSTEMS, INC.
To: TCS TALENTS, LLC, AS THE COLLATERAL AGENT
Reel/Frame 050255/0071 →
RELEASE OF SECURITY INTEREST Recorded Sep 3, 2019
From: ORIX GROWTH CAPITAL, LLC
To: SMARTDRIVE SYSTEMS, INC.
Reel/Frame 050254/0872 →
RELEASE OF IP SECURITY AGREEMENT Recorded Sep 3, 2019
From: ALLY BANK
To: SMARTDRIVE SYSTEMS, INC.
Reel/Frame 050258/0222 →
SECURITY INTEREST Recorded Feb 15, 2019
From: SMARTDRIVE SYSTEMS, INC.
To: ORIX GROWTH CAPITAL, LLC
Reel/Frame 048352/0843 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Aug 26, 2016
From: SMARTDRIVE SYSTEMS, INC.
To: ALLY BANK
Reel/Frame 039841/0148 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 13, 2014
From: PALMER, JASON; SLJIVAR, SLAVEN; FREITAS, MARK; DENINGER, DANIEL A.; GRISWOLD, JEFFREY TODD
To: SMARTDRIVE SYSTEMS, INC.
Reel/Frame 034167/0505 →
Cited By (13)
US 12,534,097 US 12,561,624 US 12,565,143 US 12,626,200 US 12,630,050 US 12,646,402 US 12,651,529 US 12,662,152 US 12,665,989 US 12,671,464 US 12,675,419 US 12,682,753 US 12,701,004