IP Library › Granted Patent US 12,254,067
Granted Patent B2
US 12,254,067 · App. 18/530,550 · Granted Mar 18, 2025

Synchronizing image data with either vehicle telematics data or infrastructure data pertaining to a road segment

Inventors: Alexander Cardona (Gilbert, AZ); Kip Wilson (Cave Creek, AZ); David Frank (Tempe, AZ); Phillip Michael Wilkowski (Phoenix, AZ); Nolan White (Chandler, AZ)
Assignee: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
G06F18/251B60W30/08B60W40/06B60W40/105G01C21/3461G01C21/3484G01C21/3602G05D1/0088G06F3/0484G06F16/29G06F16/54G06F16/5866G06F16/587G06N20/00G06Q10/20G06Q50/26G06T7/20G06T7/292G06T7/70G06V20/52G06V20/54G06V20/56G06V20/58G06V20/584G06V20/588G07C5/008G08G1/0112G08G1/052G11B27/34H04N7/18B60W2030/082B60W2420/403B60W2420/408B60W2520/10B60W2552/20B60W2552/53G01C21/3815G01C21/3848G06Q10/10G06Q40/08G06T2207/30252H04L67/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,254,067
App. No.
18/530,550
Filed
Dec 6, 2023
Granted
Mar 18, 2025
Kind
B2
Art Unit
2481
USPC
348/149
Abstract

Techniques for collecting, synchronizing, and displaying various types of data relating to a road segment enable, via one or more local or remote processors, servers, transceivers, and/or sensors, (i) enhanced and contextualized analysis of vehicle events by way of synchronizing different data types, relating to a monitored road segment, collected via various different types of data sources; (ii) enhanced and contextualized analysis of filed insurance claims pertaining to a vehicle incident at a road segment; (iii) advantageous machine learning techniques for predicting a level of risk assumed for a given vehicle event or a given road segment; (iv) techniques for accounting for region-specific driver profiles when controlling autonomous vehicles; and/or (v) improved techniques for providing a GUI to display collected data in a meaningful and contextualized manner.

Claims (48)

1. A computer-implemented method for synchronizing road segment parameters with images of a road segment to facilitate improved analysis of information about the road segment, the method comprising:

capturing, via an image sensor, a set of images of a road segment during a time-period;

detecting, by one or more processors, a plurality of road segment parameters associated with the road segment during the time-period;

time-synchronizing, by the one or more processors, the set of images and the plurality of road segment parameters according to a common clock based on an occurrence of a common event represented in the set of images and the plurality of road segment parameters;

storing, by the one or more processors, the set of images and the plurality of road segment parameters to be referenceable by a set of timestamps; and

analyzing, by the one or more processors, the set of images and the plurality of road segment parameters to identify one or more vehicle events, each characterized by a subset of the set of images and a subset of the plurality of road segment parameters.

2. The computer-implemented method of claim 1 , further comprising:

timestamping, by the one or more processors, the set of images according to a first clock associated with the image sensor and the plurality of road segment parameters according to a second clock associated with one or more vehicle sensors or one or more infrastructure devices disposed proximate to the road segment.

3. The computer-implemented method of claim 2 , wherein time-synchronizing the set of images and the plurality of road segment parameters comprises:

analyzing the set of images to identify: (i) at least one image corresponding to the occurrence of the common event, and (ii) a first time period, according to the first clock associated with the image sensor, during which the common event took place;

analyzing the plurality of road segment parameters to identify: (i) at least one road segment parameter in the plurality of road segment parameters corresponding to the occurrence of the common event, and (ii) a second time period, according to the second clock associated with the one or more vehicle sensors or the one or more infrastructure devices, during which the common event took place; and

setting the set of timestamps for the plurality of road segment parameters according to the common clock such that a first subset of the set of timestamps for the one or more images matches a subset of the set of timestamps for the plurality of road segment parameters with respect to a time at which the common event took place.

4. The computer-implemented method of claim 1 , wherein detecting the plurality of road segment parameters comprises detecting both a first set of road segment parameters collected via one or more vehicle sensors and detecting a second set of road segment parameters collected via one or more infrastructure devices disposed proximate to the road segment.

5. The computer-implemented method of claim 1 , wherein the plurality of road segment parameters comprises a parameter representing a position, a heading, or a speed of a vehicle in which one or more vehicle sensors are disposed.

6. The computer-implemented method of claim 1 , wherein the plurality of road segment parameters includes a first position for a first vehicle and a second position for a second vehicle, and

wherein the method further comprises calculating a distance between the first and second vehicles based upon the first and second positions.

7. The computer-implemented method of claim 1 , wherein the plurality of road segment parameters comprises a health status parameter representing a degree of health for an infrastructure component.

8. The computer-implemented method of claim 1 , wherein the plurality of road segment parameters comprises an operational status of an infrastructure component.

9. The computer-implemented method of claim 8 , wherein the infrastructure component is a traffic signal and wherein the operational status indicates that the traffic signal was green, yellow, or red at a given time.

10. The computer-implemented method of claim 1 , wherein the plurality of road segment parameters is collected by a radar device or a motion sensing device.

11. The computer-implemented method of claim 1 , wherein the plurality of road segment parameters includes one or more parameters indicating an atmospheric condition at the road segment.

12. A system for synchronizing road segment parameters with images of a road segment to facilitate improved analysis or display of information about the road segment, the system comprising:

an image sensor configured to capture images of a road segment; and

a computer system coupled to the image sensor, the computer system including one or more processors and computer readable instructions that, when executed by the one or more processors, cause the computer system to:

capture, via the image sensor, a set of images of the road segment during a time-period,

receive a plurality of road segment parameters associated with the road segment during the time-period,

time-synchronize the set of images and the plurality of road segment parameters according to a common clock based on an occurrence of a common event represented in the set of images and the plurality of road segment parameters,

store the set of images and the plurality of road segment parameters to be referenceable by a set of timestamps, and

analyze the set of images and the plurality of road segment parameters to identify one or more vehicle events, each characterized by a subset of the set of images and a subset of the plurality of road segment parameters.

13. The system of claim 12 , wherein the instructions, when executed, further cause the computer system to:

timestamp the set of images according to a first clock associated with the image sensor and the plurality of road segment parameters according to a second clock associated with one or more vehicle sensors or one or more infrastructure devices disposed proximate to the road segment.

14. The system of claim 13 , wherein causing the computer system to time-synchronize the set of images and the plurality of road segment parameters comprises causing the computer system to:

analyze the set of images to identify: (i) at least one image corresponding to the occurrence of the common event, and (ii) a first time period, according to the first clock associated with the image sensor, during which the common event took place;

analyze the plurality of road segment parameters to identify: (i) at least one road segment parameter in the plurality of road segment parameters corresponding to the occurrence of the common event, and (ii) a second time period, according to the second clock associated with the one or more vehicle sensors or the one or more infrastructure devices, during which the common event took place; and

set the set of timestamps for the plurality of road segment parameters according to the common clock such that a first subset of the set of timestamps for the one or more images matches a subset of the set of timestamps for the plurality of road segment parameters with respect to a time at which the common event took place.

15. The system of claim 12 , wherein the plurality of road segment parameters comprises both a first set of parameters collected by one or more vehicle sensors and a second set of parameters collected by one or more infrastructure devices disposed proximate to the road segment.

16. The system of claim 12 , wherein the plurality of road segment parameters comprises a parameter representing a position, a heading, or a speed of a vehicle in which one or more vehicle sensors are disposed.

17. The system of claim 12 , wherein the plurality of road segment parameters includes a first position for a first vehicle and a second position for a second vehicle, and

wherein the computer system further includes computer readable instructions to cause the computer system to calculate a distance between the first and second vehicles based upon the first and second positions.

18. The system of claim 12 , wherein the plurality of road segment parameters comprises: (i) a health status parameter representing a degree of health for an infrastructure component or (ii) an operational status of the infrastructure component, and

wherein the infrastructure component is a radar device or a motion sensing device.

19. The system of claim 12 , wherein the plurality of road segment parameters comprises one or more parameters indicating an atmospheric condition at the road segment.

20. A tangible machine-readable medium comprising instructions for synchronizing road segment parameters with images of a road segment to facilitate improved analysis or display of information about the road segment that, when executed, cause a machine to at least:

receive a set of images of a road segment during a time-period;

detect a plurality of road segment parameters associated with the road segment during the time-period;

time-synchronize the set of images and the plurality of road segment parameters according to a common clock based on an occurrence of a common event represented in the set of images and the plurality of road segment parameters;

store the set of images and the plurality of road segment parameters to be referenceable by a set of timestamps; and

analyze the set of images and the plurality of road segment parameters to identify one or more vehicle events, each characterized by a subset of the set of images and a subset of the plurality of road segment parameters.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 8, 2023
From: CARDONA, ALEXANDER; WILSON, KIP; FRANK, DAVID; WILKOWSKI, PHILLIP MICHAEL; WHITE, NOLAN
To: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
Reel/Frame 065809/0525 →
Continuity (5)
Continuation 17980788 · Nov 4, 2022
Continuation 17088402 · Nov 3, 2020
Provisional Application 63028732 · May 22, 2020
Provisional Application 63027628 · May 20, 2020
Related Publication 20240104171A1 · Mar 28, 2024
References Cited (67)
US 7692724B2 · Arora et al. · 2010 [cited by applicant]
US 7843974B2 · Hannuksela · 2010 [cited by applicant]
US 8189679B2 · Tsukagoshi et al. · 2012 [cited by applicant]
US 8238726B2 · Takatsuji et al. · 2012 [cited by applicant]
US 8301007B2 · Ichimura · 2012 [cited by applicant]
US 8729446B2 · Verfuerth · 2014 [cited by applicant]
US 9311271B2 · Wright · 2016 [cited by applicant]
US 9371099B2 · Lagassey · 2016 [cited by applicant]
US 9633487B2 · Wright · 2017 [cited by applicant]
US 9633560B1 · Gao et al. · 2017 [cited by applicant]
US 9667693B2 · Schramm et al. · 2017 [cited by applicant]
US 9773281B1 · Hanson · 2017 [cited by applicant]
US 9886841B1 · Nave et al. · 2018 [cited by applicant]
US 9910443B1 · Lee et al. · 2018 [cited by applicant]
US 10106156B1 · Nave et al. · 2018 [cited by applicant]
US 10178427B2 · Huang · 2019 [cited by applicant]
US 10192369B2 · Wright · 2019 [cited by applicant]
US 10198879B2 · Wright · 2019 [cited by applicant]
US 10269075B2 · Hsu-Hoffman et al. · 2019 [cited by applicant]
US 10354230B1 · Hanson et al. · 2019 [cited by applicant]
US 10719966B1 · Davis et al. · 2020 [cited by applicant]
US 10761542B1 · Fairfield et al. · 2020 [cited by applicant]
US 10977939B2 · Zhang et al. · 2021 [cited by applicant]
US 11151870B2 · Xu · 2021 [cited by applicant]
US 11270583B2 · Kumar · 2022 [cited by applicant]
US 20030093187A1 · Walker · 2003 [cited by applicant]
US 20060095199A1 · Lagassey · 2006 [cited by applicant]
US 20110015971A1 · Hembury · 2011 [cited by applicant]
US 20110161116A1 · Peak et al. · 2011 [cited by applicant]
US 20110173015A1 · Chapman et al. · 2011 [cited by applicant]
US 20110320492A1 · Inghelbrecht · 2011 [cited by applicant]
US 20120123806A1 · Schumann, Jr. et al. · 2012 [cited by applicant]
US 20140046701A1 · Steinberg et al. · 2014 [cited by applicant]
US 20140136414A1 · Abhyanker · 2014 [cited by applicant]
US 20140222321A1 · Petty et al. · 2014 [cited by applicant]
US 20140257637A1 · Sangameswaran et al. · 2014 [cited by applicant]
US 20160189305A1 · Bogovich et al. · 2016 [cited by applicant]
US 20170069001A1 · Scofield · 2017 [cited by examiner]
US 20170221149A1 · Hsu-Hoffman et al. · 2017 [cited by applicant]
US 20170267238A1 · Mimura et al. · 2017 [cited by applicant]
US 20190043273A1 · Cordova et al. · 2019 [cited by applicant]
US 20190164419A1 · Kumar · 2019 [cited by applicant]
US 20190362198A1 · Averbuch · 2019 [cited by examiner]
US 20200148200A1 · Lerner · 2020 [cited by applicant]
US 20200168084A1 · Yang · 2020 [cited by examiner]
US 20200226922A1 · Hahn · 2020 [cited by examiner]
US 20200312134A1 · Zhang et al. · 2020 [cited by applicant]
US 20210142664A1 · Xu · 2021 [cited by applicant]
US 20210239478A1 · Mathai et al. · 2021 [cited by applicant]
CA 2567074C · 2013 [cited by applicant]
EP 2866230A1 · 2015 [cited by applicant]
WO WO2015058610A1 · 2015 [cited by applicant]
Google Inc., “Temporal Synchronization of Multiple Audio Signals”. Retrieved from the Internet at: <https://research.google.com/pubs/archive/42193.pdf> on Jan. 26, 2021. [cited by applicant]
Center for Computer Research in Music and Acoustics, Stanford University, “Clustering and Synchronizing Multi-Camera Video Via Landmark Cross-Correlation”. Retrieved from the Internet at: <https://ccrma.stanford.edu/˜nj… [cited by applicant]
ACM DL Digital Library, “Making a scene: alignment of complete sets of clips based on pairwise audio match”. Retrieved from the Internet at: <https://dl.acm.org/doi/10.1145/2324796.2324829> on Jan. 26, 2021. [cited by applicant]
U.S. Appl. No. 17/088,402, Synchronizing Image Data With Either Vehicle Telematics Data or Infrastructure Data Pertaining to a Road Segment, filed Nov. 3, 2020. [cited by applicant]
U.S. Appl. No. 17/088,412, Analyzing Insurance Claims in Light of Detected Conditions Pertaining to a Road Segment, filed Nov. 3, 2020. [cited by applicant]
U.S. Appl. No. 17/088,417, Risk Analysis for Vehicle Events and Road Segments Based On Detected Conditions Pertaining to the Road, filed Nov. 3, 2020. [cited by applicant]
U.S. Appl. No. 17/088,424, Training a Machine-Learning Model Using Detected Conditions at a Road Segment, filed Nov. 3, 2020. [cited by applicant]
U.S. Appl. No. 17/088,430, Providing a Gui to Enable Analysis of Time-Synchronized Data Sets Pertaining to a Road Segment, filed Nov. 3, 2020. [cited by applicant]
U.S. Appl. No. 17/088,434, Accounting for Region-Specific Driver Profiles When Controlling Autonomous Vehicles, filed Nov. 3, 2020. [cited by applicant]
U.S. Appl. No. 17/088,393, Synchronizing Vehicle Telematics Data With Infrastructure Data Pertaining to a Road Segment, filed Nov. 3, 2020. [cited by applicant]
School of Computer Science, Carnegie Mellon University, “Synchronization for Multi-Perspective Videos in the Wild” Retrieved from the Internet at: <http://www.cs.cmu.edu/˜poyaoh/data/ICASSP_2017.pdf> on Oct. 6, 2020. [cited by applicant]
Google Inc., “Temporal Synchronization of Multiple Audio Signals”. Retrieved from the Internet at: <https://research.google.com/pubs/archive/42193.pdf> dated Jan. 26, 2021. [cited by applicant]
Center for Computer Research in Music and Acoustics, Stanford University, “Clustering and Synchronizing Multi-Camera Video Via Landmark Cross-Correlation”. Retrieved from the Internet at: <https://ccrma.stanford.edu/˜nj… [cited by applicant]
School of Computer Science, Carnegie Mellon University, “Synchronization for Multi-Perspective Videos in the Wild” Retrieved from the Internet: <http://www.cs.cmu.edu/˜poyaoh/data/ICASSP_2017.pdf> on Oct. 6, 2020. [cited by applicant]
ACM DL Digital Library, “Making a scene: alignment of complete sets of clips based on pairwise audio match”. Retrieved from the Internet at: <https://dl.acm.org/doi/10.1145/2324796.2324829> dated Jan. 26, 2021. [cited by applicant]