IP Library Granted Patent US 12,321,421
Granted Patent B2
US 12,321,421 · App. 18/201,027 · Granted Jun 3, 2025

Providing a GUI to enable analysis of time-synchronized data sets pertaining to a road segment

Inventors: Alexander Cardona (Gilbert, AZ); Kip Wilson (Cave Creek, AZ); David Frank (Tempe, AZ); Phillip Michael Wilkowski (Phoenix, AZ); Nolan White (Chandler, AZ)
Assignee: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
G06F18/251B60W30/08B60W30/182B60W40/06B60W40/105G01C21/3461G01C21/3484G01C21/3602G05D1/0088G06F3/0484G06F16/29G06F16/54G06F16/5866G06F16/587G06N20/00G06Q10/20G06Q50/26G06T7/20G06T7/292G06T7/70G06V20/52G06V20/54G06V20/56G06V20/58G06V20/584G06V20/588G07C5/008G08G1/0112G08G1/052G11B27/34H04N7/18H04W4/44B60W2030/082B60W2420/403B60W2420/408B60W2520/10B60W2552/20B60W2552/53B60W2556/50B60W2556/55G01C21/3815G01C21/3848G06Q10/10G06Q40/08G06T2207/30252H04L67/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,321,421
App. No.
18/201,027
Granted
Jun 3, 2025
Kind
B2
Abstract

Techniques for collecting, synchronizing, and displaying various types of data relating to a road segment enable, via one or more local or remote processors, servers, transceivers, and/or sensors, (i) enhanced and contextualized analysis of vehicle events by way of synchronizing different data types, relating to a monitored road segment, collected via various different types of data sources; (ii) enhanced and contextualized analysis of filed insurance claims pertaining to a vehicle incident at a road segment; (iii) advantageous machine learning techniques for predicting a level of risk assumed for a given vehicle event or a given road segment; (iv) techniques for accounting for region-specific driver profiles when controlling autonomous vehicles; and/or (v) improved techniques for providing a GUI to display collected data in a meaningful and contextualized manner.

Claims (41)

1. A computer-implemented method providing a graphic user interface to facilitate analyzing vehicle events at a road segment, the method comprising, via one or more local or remote processors, servers, transceivers, and/or sensors:

(A) displaying a graphic user interface (GUI) configured to display at least one of a series of images, the GUI including an image control element interactable to advance forward or backward in time through the series of images;

(B) analyzing a displayed image from the series of images displayed within the GUI to identify a timestamp for the displayed image;

(C) retrieving values for a set of road segment parameters based upon the identified timestamp, such that the values for the set of road segment parameters are relevant-in-time to the displayed image, wherein the set of road segment parameters comprise a parameter collected via one or more vehicle sensors and a parameter collected via one or more infrastructure devices;

(D) dynamically calculating one or more risk indices by analyzing the set of road segment parameters using a machine learning model previously trained on synchronized training data including telematics data, contextual data regarding vehicle operating environments that is synchronized with the telematics data, and vehicle incident data from a plurality of past vehicle events;

(E) analyzing the set of road segment parameters to generate a behavioral characterization of one or more drivers; and

(F) displaying the behavioral characterization of the one or more drivers simultaneous to the displaying of the displayed image, such that the behavioral characterization, relevant-in-time to the displayed image, is simultaneously viewable with the displayed image.

2. The computer-implemented method of claim 1 , wherein the parameter collected via the one or more vehicle sensors represents a speed, position, or heading of a vehicle.

3. The computer-implemented method of claim 1 , wherein the parameter collected via the one or more vehicle sensors represents a proximity between a first vehicle in which the one or more vehicle sensors are disposed and a second vehicle detected via the one or more vehicle sensors.

4. The computer-implemented method of claim 1 , wherein the parameter collected via the one or more infrastructure devices represents an atmospheric condition.

5. The computer-implemented method of claim 1 , wherein the parameter collected via the one or more infrastructure devices is a health status parameter representing a degree of health for an infrastructure component.

6. The computer-implemented method of claim 1 , wherein the parameter collected via the one or more infrastructure devices is an operational status parameter representing an operational status of an infrastructure component.

7. The computer-implemented method of claim 1 , wherein the parameter collected via the one or more infrastructure devices is a speed collected via a radar device.

8. A system for providing a graphic user interface to facilitate analyzing vehicle events at a road segment, the system comprising:

a display;

one or more processors coupled to the display;

a memory coupled to the one or more processors and storing computer-readable instructions that, when executed, cause the one or more processors to:

(A) display a graphic user interface (GUI) configured to display at least one of a series of images, the GUI including an image control element interactable to advance forward or backward in time through the series of images;

(B) analyze a displayed image from the series of images displayed within the GUI to identify a timestamp for the displayed image;

(C) retrieve values for a set of road segment parameters based upon the identified timestamp, such that the values for the set of road segment parameters are relevant-in-time to the displayed image, wherein the set of road segment parameters comprise a parameter collected via one or more vehicle sensors and a parameter collected via one or more infrastructure devices;

(D) dynamically calculate one or more risk indices by analyzing the set of road segment parameters using a machine learning model previously trained on synchronized training data including telematics data, contextual data regarding vehicle operating environments that is synchronized with the telematics data, and vehicle incident data from a plurality of past vehicle events;

(E) analyze the set of road segment parameters to generate a behavioral characterization of one or more drivers; and

(F) display the behavioral characterization of the one or more drivers simultaneous to the displaying of the displayed image, such that the behavioral characterization, relevant-in-time to the displayed image, is simultaneously viewable with the displayed image.

9. The system of claim 8 , wherein the parameter collected via the one or more vehicle sensors represents a speed, position, or heading of a vehicle.

10. The system of claim 8 , wherein the parameter collected via the one or more vehicle sensors represents a proximity between a first vehicle in which the one or more vehicle sensors are disposed and a second vehicle detected via the one or more vehicle sensors.

11. The system of claim 8 , wherein the parameter collected via the one or more infrastructure devices represents an atmospheric condition.

12. The system of claim 8 , wherein the parameter collected via the one or more infrastructure devices is a health status parameter representing a degree of health for an infrastructure component.

13. The system of claim 8 , wherein the parameter collected via the one or more infrastructure devices is an operational status parameter representing an operational status of an infrastructure component.

14. The system of claim 8 , wherein the parameter collected via the one or more infrastructure devices is a speed collected via a radar device.

15. A tangible, non-transitory computer-readable medium storing executable instructions that, when executed by one or more processors of a computer system, cause the computer system to:

(A) display a graphic user interface (GUI) configured to display at least one of a series of images, the GUI including an image control element interactable to advance forward or backward in time through the series of images;

(B) analyze a displayed image from the series of images displayed within the GUI to identify a timestamp for the displayed image;

(C) retrieve values for a set of road segment parameters based upon the identified timestamp, such that the values for the set of road segment parameters are relevant-in-time to the displayed image, wherein the set of road segment parameters comprise a parameter collected via one or more vehicle sensors and a parameter collected via one or more infrastructure devices;

(D) dynamically calculate one or more risk indices by analyzing the set of road segment parameters using a machine learning model previously trained on synchronized training data including telematics data, contextual data regarding vehicle operating environments that is synchronized with the telematics data, and vehicle incident data from a plurality of past vehicle events;

(E) analyze the set of road segment parameters to generate a behavioral characterization of one or more drivers; and

(F) display the behavioral characterization of the one or more drivers simultaneous to the displaying of the displayed image, such that the behavioral characterization, relevant-in-time to the displayed image, is simultaneously viewable with the displayed image.

16. The tangible, non-transitory computer-readable medium of claim 15 , wherein the parameter collected via the one or more vehicle sensors represents a speed, position, or heading of a vehicle.

17. The tangible, non-transitory computer-readable medium of claim 15 , wherein the parameter collected via the one or more vehicle sensors represents a proximity between a first vehicle in which the one or more vehicle sensors are disposed and a second vehicle detected via the one or more vehicle sensors.

18. The tangible, non-transitory computer-readable medium of claim 15 , wherein the parameter collected via the one or more infrastructure devices represents an atmospheric condition.

19. The tangible, non-transitory computer-readable medium of claim 15 , wherein the parameter collected via the one or more infrastructure devices is a health status parameter representing a degree of health for an infrastructure component.

20. The tangible, non-transitory computer-readable medium of claim 15 , wherein the parameter collected via the one or more infrastructure devices is an operational status parameter representing an operational status of an infrastructure component.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 31, 2023
From: CARDONA, ALEXANDER; WILSON, KIP; FRANK, DAVID; WILKOWSKI, PHILLIP MICHAEL; WHITE, NOLAN
To: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
Reel/Frame 063809/0263 →
Continuity (4)
Continuation 17088430 · Nov 3, 2020
Provisional Application 63028732 · May 22, 2020
Provisional Application 63027628 · May 20, 2020
Related Publication 20230377078A1 · Nov 23, 2023
References Cited (170)
US 5644611A · McShane · 1997 [cited by examiner]
US 6016438A · Wakayama · 2000 [cited by examiner]
US 6049622A · Robb · 2000 [cited by examiner]
US 6108573A · Debbins · 2000 [cited by examiner]
US 6111573A · McComb · 2000 [cited by examiner]
US 6182127B1 · Cronin, III · 2001 [cited by examiner]
US 6502114B1 · Forcier · 2002 [cited by examiner]
US 6510459B2 · Cronin, III · 2003 [cited by examiner]
US 7034860B2 · Lia · 2006 [cited by examiner]
US 7389178B2 · Raz et al. · 2008 [cited by applicant]
US 7417644B2 · Cooper · 2008 [cited by examiner]
US 7647338B2 · Lazier · 2010 [cited by examiner]
US 7676763B2 · Rummel · 2010 [cited by examiner]
US 7692724B2 · Arora et al. · 2010 [cited by applicant]
US 7730425B2 · de los Reyes · 2010 [cited by examiner]
US 7792328B2 · Albertson et al. · 2010 [cited by applicant]
US 7843974B2 · Hannuksela · 2010 [cited by applicant]
US 8189679B2 · Tsukagoshi et al. · 2012 [cited by applicant]
US 8238726B2 · Takatsuji et al. · 2012 [cited by applicant]
US 8245156B2 · Mouilleseaux · 2012 [cited by examiner]
US 8279174B2 · Jee · 2012 [cited by examiner]
US 8301007B2 · Ichimura · 2012 [cited by applicant]
US 8381133B2 · Iwema · 2013 [cited by examiner]
US 8549432B2 · Warner · 2013 [cited by examiner]
US 8577703B2 · McClellan et al. · 2013 [cited by applicant]
US 8601389B2 · Schulz · 2013 [cited by examiner]
US 8630768B2 · McClellan et al. · 2014 [cited by applicant]
US 8700255B2 · Joseph · 2014 [cited by applicant]
US 8729446B2 · Verfuerth · 2014 [cited by applicant]
US 8996987B2 · Rivera · 2015 [cited by examiner]
US 9311271B2 · Wright · 2016 [cited by applicant]
US 9371099B2 · Lagassey · 2016 [cited by examiner]
US 9633487B2 · Wright · 2017 [cited by applicant]
US 9633560B1 · Gao · 2017 [cited by examiner]
US 9639804B1 · Palmer et al. · 2017 [cited by applicant]
US 9667693B2 · Schramm et al. · 2017 [cited by applicant]
US 9734532B1 · Brandmaier · 2017 [cited by examiner]
US 9773281B1 · Hanson · 2017 [cited by applicant]
US 9830748B2 · Rosenbaum · 2017 [cited by applicant]
US 9881340B2 · Kenefick et al. · 2018 [cited by applicant]
US 9886841B1 · Nave et al. · 2018 [cited by applicant]
US 9910443B1 · Lee et al. · 2018 [cited by applicant]
US 9990782B2 · Rosenbaum · 2018 [cited by applicant]
US 10106156B1 · Nave et al. · 2018 [cited by applicant]
US 10178427B2 · Huang · 2019 [cited by applicant]
US 10192369B2 · Wright · 2019 [cited by applicant]
US 10198879B2 · Wright · 2019 [cited by applicant]
US 10269075B2 · Hsu-Hoffman et al. · 2019 [cited by applicant]
US 10269190B2 · Rosenbaum · 2019 [cited by applicant]
US 10354230B1 · Hanson et al. · 2019 [cited by applicant]
US 10467824B2 · Rosenbaum · 2019 [cited by applicant]
US 10719966B1 · Davis et al. · 2020 [cited by applicant]
US 10761542B1 · Fairfield · 2020 [cited by examiner]
US 10789840B2 · Boykin et al. · 2020 [cited by applicant]
US 10977939B2 · Zhang et al. · 2021 [cited by applicant]
US 11062341B2 · Ramirez et al. · 2021 [cited by applicant]
US 11151870B2 · Xu · 2021 [cited by applicant]
US 11227452B2 · Rosenbaum · 2022 [cited by applicant]
US 11270583B2 · Kumar · 2022 [cited by applicant]
US 11407410B2 · Rosenbaum · 2022 [cited by applicant]
US 11524707B2 · Rosenbaum · 2022 [cited by applicant]
US 11526711B1 · Cardona · 2022 [cited by examiner]
US 11594083B1 · Rosenbaum · 2023 [cited by applicant]
US 20030011638A1 · Chung · 2003 [cited by examiner]
US 20030093187A1 · Walker · 2003 [cited by examiner]
US 20040230940A1 · Cooper · 2004 [cited by examiner]
US 20050091596A1 · Anthony · 2005 [cited by examiner]
US 20050228250A1 · Bitter · 2005 [cited by examiner]
US 20050251021A1 · Kaufman · 2005 [cited by examiner]
US 20050278647A1 · Leavitt · 2005 [cited by examiner]
US 20060013462A1 · Sadikali · 2006 [cited by examiner]
US 20060095199A1 · Lagassey · 2006 [cited by examiner]
US 20070130519A1 · Tilford · 2007 [cited by examiner]
US 20070176898A1 · Suh · 2007 [cited by examiner]
US 20070180397A1 · Hoyer · 2007 [cited by examiner]
US 20080036743A1 · Westerman · 2008 [cited by examiner]
US 20080201315A1 · Lazier · 2008 [cited by examiner]
US 20080218533A1 · Goto · 2008 [cited by examiner]
US 20090144741A1 · Tsuda · 2009 [cited by examiner]
US 20090309849A1 · Iwema · 2009 [cited by examiner]
US 20090327964A1 · Mouilleseaux · 2009 [cited by examiner]
US 20100174449A1 · Kim · 2010 [cited by applicant]
US 20100192103A1 · Cragun · 2010 [cited by examiner]
US 20110015971A1 · Hembury · 2011 [cited by examiner]
US 20110154268A1 · Trent, Jr. · 2011 [cited by examiner]
US 20110161116A1 · Peak · 2011 [cited by examiner]
US 20110173015A1 · Chapman et al. · 2011 [cited by applicant]
US 20110218827A1 · Kenefick et al. · 2011 [cited by applicant]
US 20110252346A1 · Chaudhri · 2011 [cited by examiner]
US 20110320492A1 · Inghelbrecht · 2011 [cited by applicant]
US 20120013540A1 · Hogan · 2012 [cited by examiner]
US 20120123806A1 · Schumann, Jr. · 2012 [cited by examiner]
US 20120136905A1 · Pullara · 2012 [cited by examiner]
US 20120278725A1 · Gordon · 2012 [cited by examiner]
US 20130067391A1 · Pittappilly · 2013 [cited by examiner]
US 20130091471A1 · Gutt · 2013 [cited by examiner]
US 20140025225A1 · Armitage et al. · 2014 [cited by applicant]
US 20140046701A1 · Steinberg et al. · 2014 [cited by applicant]
US 20140058761A1 · Freiberger et al. · 2014 [cited by applicant]
US 20140075286A1 · Harada · 2014 [cited by examiner]
US 20140136414A1 · Abhyanker · 2014 [cited by examiner]
US 20140143839A1 · Ricci · 2014 [cited by examiner]
US 20140222321A1 · Petty et al. · 2014 [cited by applicant]
US 20140257637A1 · Sangameswaran et al. · 2014 [cited by applicant]
US 20140279707A1 · Joshua · 2014 [cited by examiner]
US 20140303961A1 · Leydon · 2014 [cited by examiner]
US 20140306826A1 · Ricci · 2014 [cited by examiner]
US 20150309512A1 · Cudak et al. · 2015 [cited by applicant]
US 20160092962A1 · Wasserman · 2016 [cited by examiner]
US 20160189305A1 · Bogovich et al. · 2016 [cited by applicant]
US 20160267723A1 · Rajpathak · 2016 [cited by examiner]
US 20170069001A1 · Scofield et al. · 2017 [cited by applicant]
US 20170191847A1 · Chintakindi et al. · 2017 [cited by applicant]
US 20170221149A1 · Hsu-Hoffman et al. · 2017 [cited by applicant]
US 20170267238A1 · Mimura · 2017 [cited by examiner]
US 20170270612A1 · Howe et al. · 2017 [cited by applicant]
US 20180025430A1 · Perl · 2018 [cited by examiner]
US 20180122234A1 · Nascimento et al. · 2018 [cited by applicant]
US 20180208195A1 · Hutcheson et al. · 2018 [cited by applicant]
US 20180339653A1 · Adams et al. · 2018 [cited by applicant]
US 20180365772A1 · Thompson et al. · 2018 [cited by applicant]
US 20190043273A1 · Cordova et al. · 2019 [cited by applicant]
US 20190164419A1 · Kumar · 2019 [cited by applicant]
US 20190265712A1 · Satzoda · 2019 [cited by examiner]
US 20190362198A1 · Averbuch et al. · 2019 [cited by applicant]
US 20190378201A1 · Briggs · 2019 [cited by examiner]
US 20200041612A1 · Harrison · 2020 [cited by examiner]
US 20200148200A1 · Lerner · 2020 [cited by applicant]
US 20200160704A1 · Choi et al. · 2020 [cited by applicant]
US 20200168084A1 · Yang et al. · 2020 [cited by applicant]
US 20200210336A1 · Bräutigam et al. · 2020 [cited by applicant]
US 20200226922A1 · Hahn et al. · 2020 [cited by applicant]
US 20200312134A1 · Zhang et al. · 2020 [cited by applicant]
US 20200334762A1 · Carver · 2020 [cited by examiner]
US 20210082054A1 · Rodriguez Bravo et al. · 2021 [cited by applicant]
US 20210142664A1 · Xu · 2021 [cited by applicant]
US 20210239478A1 · Mathai et al. · 2021 [cited by applicant]
US 20210312561A1 · Speasl et al. · 2021 [cited by applicant]
US 20220092893A1 · Rosenbaum · 2022 [cited by applicant]
US 20220126863A1 · Moustafa et al. · 2022 [cited by applicant]
US 20220340148A1 · Rosenbaum · 2022 [cited by applicant]
US 20230060300A1 · Rosenbaum · 2023 [cited by applicant]
US 20230113815A1 · Patt et al. · 2023 [cited by applicant]
CA 2567074C · 2013 [cited by applicant]
EP 2053497A1 · 2009 [cited by examiner]
EP 2866230A1 · 2015 [cited by applicant]
EP 3239686A1 · 2017 [cited by applicant]
EP 3578433B1 · 2020 [cited by applicant]
EP 3730375B1 · 2021 [cited by applicant]
EP 3960576A1 · 2022 [cited by applicant]
EP 4190659A1 · 2023 [cited by applicant]
EP 4190660A1 · 2023 [cited by applicant]
JP 2004280348A · 2004 [cited by applicant]
WO WO2015058610A1 · 2015 [cited by applicant]
Google Inc., “Temporal Synchronization of Multiple Audio Signals”. Retrieved from the Internet at: <https://research.google.com/pubs/archive/42193.pdf> on Jan. 26, 2021. [cited by applicant]
Center for Computer Research in Music and Acoustics, Stanford University, “Clustering and Synchronizing Multi-Camera Video via Landmark Cross-Correlation”. Retrieved from the Internet at: <https://ccrma.stanford.edu/˜nj… [cited by applicant]
ACM DL Digital Library, “Making a scene: alignment of complete sets of clips based on pairwise audio match”. Retrieved from the Internet at: <https://dl.acm.org/doi/10.1145/2324796.2324829> on Jan. 26, 2021. [cited by applicant]
U.S. Appl. No. 17/088,402, Synchronizing Image Data With Either Vehicle Telematics Data or Infrastructure Data Pertaining to a Road Segment, filed Nov. 3, 2020. [cited by applicant]
U.S. Appl. No. 17/088,412, Analyzing Insurance Claims in Light of Detected Conditions Pertaining to a Road Segment, filed Nov. 3, 2020. [cited by applicant]
U.S. Appl. No. 17/088,417, Risk Analysis for Vehicle Events and Road Segments Based on Detected Conditions Pertaining to the Road, filed Nov. 3, 2020. [cited by applicant]
U.S. Appl. No. 17/088,424, Training a Machine-Learning Model Using Detected Conditions at a Road Segment, filed Nov. 3, 2020. [cited by applicant]
U.S. Appl. No. 17/088,430, Providing a GUI to Enable Analysis of Time-Synchronized Data Sets Pertaining to a Road Segment, filed Nov. 3, 2020. [cited by applicant]
U.S. Appl. No. 17/088,434, Accounting for Region-Specific Driver Profiles When Controlling Autonomous Vehicles, filed Nov. 3, 2020. [cited by applicant]
U.S. Appl. No. 17/088,393, Synchronizing Vehicle Telematics Data With Infrastructure Data Pertaining to a Road Segment, filed Nov. 3, 2020. [cited by applicant]
School of Computer Science, Carnegie Mellon University, “Synchronization for Multi-Perspective Videos in the Wild” Retrieved from the Internet at: <http://www.cs.cmu.edu/˜poyaoh/data/ICASSP_2017.pdf> on Oct. 6, 2020. [cited by applicant]
Google Inc., “Temporal Synchronization of Multiple Audio Signals”. Retrieved from the Internet at: <https://research.google.com/pubs/archive/42193.pdf> dated Jan. 26, 2021. [cited by applicant]
Center for Computer Research in Music and Acoustics, Stanford University, “Clustering and Synchronizing Multi-Camera Video via Landmark Cross-Correlation”. Retrieved from the Internet at: <https://ccrma.stanford.edu/˜nj… [cited by applicant]
School of Computer Science, Carnegie Mellon University, “Synchronization for Multi-Perspective Videos in the Wild” Retrieved from the Internet: <http://www.cs.cmu.edu/˜poyaoh/data/ICASSP_2017.pdf> on Oct. 6, 2020. [cited by applicant]
ACM DL Digital Library, “Making a scene: alignment of complete sets of clips based on pairwise audio match”. Retrieved from the Internet at: <https://dl.acm.org/doi/10.1145/2324796.2324829> dated Jan. 26, 2021. [cited by applicant]
English Translation of “JP2004280348 A, Kurata et al., 2004” (Year: 2004). [cited by applicant]