IP Library Granted Patent US 12,307,384
Granted Patent B1
US 12,307,384 · App. 19/022,090 · Granted May 20, 2025

Systems and methods of sensor data fusion

Inventor: Armando Montalvo (Winter Garden, FL)
Assignee: Digital Global Systems, Inc.
G06N5/022
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,307,384
App. No.
19/022,090
Granted
May 20, 2025
Kind
B1
Abstract

Systems and methods of sensor data fusion including sensor data capture, curation, linking, fusion, inference, and validation. The systems and methods described herein reduce computational demand and processing time by curating data and calculating conditional entropy. The system is operable to fuse data from a plurality of sensor types. A computer processor optionally stores fused sensor data that the system validates above a mathematical threshold.

Claims (55)

1. A system for sensor data fusion for sensor management and utilization in autonomous transportation, comprising:

at least one computer processor including a memory;

at least one curation engine, at least one link engine, at least one fusion engine, at least one inference engine, and at least one validation engine;

at least one first distance sensor operable to capture a first distance measurement from a vehicle to at least one object; and

at least one second distance sensor operable to capture a second distance measurement from the vehicle to the at least one object;

wherein the at least one computer processor is operable to analyze the first distance measurement and the second distance measurement;

wherein the at least one computer processor is operable to receive at least one query;

wherein the at least one curation engine is operable to curate the first distance measurement and the second distance measurement, the at least one link engine is operable to link the first distance measurement and the second distance measurement, the at least one fusion engine is operable to fuse the first distance measurement and the second distance measurement, the at least one inference engine is operable to determine at least one inference from the first distance measurement and the second distance measurement, and the at least one validation engine is operable to validate the first distance measurement and the second distance measurement;

wherein the at least one link engine is operable to calculate a non-existent link, a weak link, or a strong link between the first distance measurement and the second distance measurement;

wherein the at least one fusion engine is operable to fuse the first distance measurement and the second distance measurement based in part on the strong link and the at least one query, thereby creating fused data;

wherein the fused data includes at least one new data set;

wherein the at least one new data set includes an accuracy value for the at least one first distance sensor and/or the at least one second distance sensor; and

wherein the at least one computer processor is operable to instruct the vehicle to brake based on the at least one new data set.

2. The system of claim 1 , wherein the at least one link engine is operable to dynamically adjust a threshold for the non-existent link, the weak link, or the strong link depending in part on the at least one query.

3. The system of claim 1 , wherein the at least one fusion engine is further operable to fuse at least one first distance property and/or at least one first distance sub-property and at least one second distance property and/or at least one second distance sub-property.

4. The system of claim 3 , wherein the at least one first distance property and/or the at least one first distance sub-property is different than the at least one second distance property and/or the at least one second distance sub-property.

5. The system of claim 1 , wherein the at least one fusion engine is operable to fuse the first distance measurement and the second distance measurement in real-time.

6. The system of claim 1 , wherein the at least one new data set further includes a prediction about a future event.

7. The system of claim 1 , wherein the at least one query is user and/or computer generated.

8. A method for sensor data fusion for sensor management and utilization in autonomous transportation, comprising:

providing at least one computer processor including a memory;

providing at least one curation engine, at least one link engine, at least one fusion engine, at least one inference engine, and at least one validation engine;

at least one first distance sensor capturing a first distance measurement from a vehicle to at least one object;

at least one second distance sensor capturing a second distance measurement from the vehicle to the at least one object;

analyzing by the at least one computer processor the first distance measurement and the second distance measurement;

receiving by the at least one computer processor at least one query;

curating by the at least one curation engine the first distance measurement and the second distance measurement, linking by the at least one link engine the first distance measurement and the second distance measurement, fusing by the at least one fusion engine the first distance measurement and the second distance measurement, determining at least one inference by the at least one inference engine from the first distance measurement and the second distance measurement, and validating by the at least one validation engine the first distance measurement and the second distance measurement;

calculating by the at least one link engine a non-existent link, a weak link, or a strong link between the first distance measurement and the second distance measurement;

fusing by the at least one fusion engine the first distance measurement and the second distance measurement based in part on the strong link and the at least one query, thereby creating fused data;

wherein the fused data includes at least one new data set;

wherein the at least one new data set includes an accuracy value for the at least one first distance sensor and/or the at least one second distance sensor; and

instructing by the at least one computer processor the vehicle to brake based on the at least one new data set.

9. The method of claim 8 , further comprising predicting a future event based in part on the fused data.

10. The method of claim 8 , further comprising dynamically adjusting via the at least one link engine a threshold for the non-existent link, the weak link, or the strong link depending in part on the at least one query.

11. The method of claim 8 , wherein the at least one query is user and/or computer generated.

12. The method of claim 8 , further comprising fusing via the at least one fusion engine at least one first distance property and/or at least one first distance sub-property and at least one second distance property and/or at least one second distance sub-property.

13. The method of claim 12 , wherein the at least one first distance property and/or the at least one first sub-property is different than the at least one second distance property and/or the at least one second distance sub-property.

14. The method of claim 8 , wherein fusing the first distance measurement and the second distance measurement occurs in real-time.

15. A system for sensor data fusion for sensor management and utilization in autonomous transportation, comprising:

at least one computer processor including a memory;

at least one curation engine, at least one link engine, at least one fusion engine, at least one inference engine, and at least one validation engine; and

at least two sensors, each of the at least two sensors operable to measure a first distance from a vehicle to at least one object and a second distance from the vehicle to the at least one object;

wherein the at least one computer processor is operable to analyze the first distance and the second distance;

wherein the at least one computer processor is operable to receive at least one query;

wherein the at least one curation engine is operable to curate the first distance and the second distance, the at least one link engine is operable to link the first distance and the second distance, the at least one fusion engine is operable to fuse the first distance and the second distance, the at least one inference engine is operable to determine at least one inference from the first distance and the second distance, and the at least one validation engine is operable to validate the first distance and the second distance;

wherein the at least one link engine is operable to iteratively calculate a non-existent link, a weak link, or a strong link between the first distance and the second distance;

wherein the at least one fusion engine is operable to fuse the first distance, the second distance, at least one first distance property and/or at least one first distance sub-property, and at least one second distance property and/or at least one second distance sub-property based in part on the strong link and/or the at least one query, thereby creating fused data;

wherein the fused data includes at least one new data set;

wherein the at least one new data set includes an accuracy value for the at least two sensors; and

wherein the at least one computer processor is operable to instruct the vehicle to move based on the at least one new data set.

16. The system of claim 15 , wherein the at least one new data set further includes a prediction about a future event.

17. The system of claim 15 , wherein the system is operable to store only the fused data.

18. The system of claim 15 , wherein the at least one fusion engine is operable to fuse the first distance, the second distance, the at least one first distance property and/or the at least one first distance sub-property, and the at least one second distance property and/or the at least one second distance sub-property in real-time.

19. The system of claim 15 , wherein the at least one first distance property and/or the at least one first distance sub-property is the same as the at least one second distance property and/or the at least one second distance sub-property.

20. The system of claim 15 , wherein the at least one first distance property and/or the at least one first distance sub-property is different than the at least one second distance property and/or the at least one second distance sub-property.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 13, 2025
From: MONTALVO, ARMANDO
To: DIGITAL GLOBAL SYSTEMS, INC.
Reel/Frame 070214/0056 →
Continuity (2)
Continuation 18990145 · Dec 20, 2024
Continuation 18988120 · Dec 19, 2024
References Cited (48)
US 9367683B2 · Kolacinski et al. · 2016 [cited by applicant]
US 10748075B2 · Chowdhary et al. · 2020 [cited by applicant]
US 10766137B1 · Porter et al. · 2020 [cited by applicant]
US 11037320B1 · Ebrahimi Afrouzi et al. · 2021 [cited by applicant]
US 11586865B2 · Brahma et al. · 2023 [cited by applicant]
US 20080312756A1 · Grichnik et al. · 2008 [cited by applicant]
US 20150306770A1 · Mittal et al. · 2015 [cited by applicant]
US 20170004411A1 · Hassan et al. · 2017 [cited by applicant]
US 20180284758A1 · Cella et al. · 2018 [cited by applicant]
US 20190258251A1 · Ditty · 2019 [cited by examiner]
US 20190295001A1 · Fusco et al. · 2019 [cited by applicant]
US 20200026289A1 · Alvarez et al. · 2020 [cited by applicant]
US 20210150230A1 · Smolyanskiy et al. · 2021 [cited by applicant]
US 20210342656A1 · Mittal et al. · 2021 [cited by applicant]
US 20220012562A1 · Lazaro-Gredilla et al. · 2022 [cited by applicant]
US 20220075077A1 · Kato · 2022 [cited by examiner]
US 20220126864A1 · Moustafa et al. · 2022 [cited by applicant]
US 20220146652A1 · Cardei et al. · 2022 [cited by applicant]
US 20220161815A1 · Van Beek et al. · 2022 [cited by applicant]
US 20220179056A1 · Braley et al. · 2022 [cited by applicant]
US 20230082656A1 · Li · 2023 [cited by examiner]
US 20230110027A1 · Bajpayee et al. · 2023 [cited by applicant]
US 20230112441A1 · Tang et al. · 2023 [cited by applicant]
US 20230166765A1 · Yoon et al. · 2023 [cited by applicant]
US 20230182768A1 · Oh · 2023 [cited by examiner]
US 20230342944A1 · Sullivan et al. · 2023 [cited by applicant]
US 20240045426A1 · Ditty et al. · 2024 [cited by applicant]
US 20240112428A1 · Levi et al. · 2024 [cited by applicant]
US 20240142994A1 · Ebrahimi Afrouzi · 2024 [cited by applicant]
US 20240152734A1 · Ye · 2024 [cited by applicant]
US 20240249165A1 · Roche et al. · 2024 [cited by applicant]
US 20240289930A1 · Yang et al. · 2024 [cited by applicant]
US 20240289981A1 · Kuo et al. · 2024 [cited by applicant]
US 20240296044A1 · Day et al. · 2024 [cited by applicant]
US 20240312219A1 · Choi et al. · 2024 [cited by applicant]
US 20240317254A1 · Tran · 2024 [cited by applicant]
US 20240324838A1 · Ebrahimi Afrouzi · 2024 [cited by applicant]
US 20240331403A1 · Shen et al. · 2024 [cited by applicant]
US 20240378412A1 · Schrader · 2024 [cited by applicant]
US 20240408495A1 · Hibi et al. · 2024 [cited by applicant]
US 20250026371A1 · Andert · 2025 [cited by examiner]
CN 110598299A · 2019 [cited by applicant]
CN 116522279A · 2023 [cited by applicant]
Yeong, D. J., Velasco-Hernandez, G., Barry, J., & Walsh, J. (2021). Sensor and sensor fusion technology in autonomous vehicles: A review. Sensors, 21(6), 2140. (Year: 2021). [cited by applicant]
Ebel, Patrick, and et al. “SEN12MS-CR-TS: A remote-sensing data set for multi modal multitemporal cloud removal.” IEEE Transactions on Geoscience and Remote Sensing 60 (2022): 1-14 (Year: 2022). [cited by applicant]
Wei Z, and et al. MmWave Radar and Vision Fusion for Object Detection in Autonomous Driving: A Review. Sensors (Basel). Mar. 25, 2022;22(7):2542. doi: 10.3390/s22072542. PMID: 35408157; PMCID: PMC9003130 (Year: 2022). [cited by applicant]
Chen et al., “Track correlation uncertainty evaluation for Multi-sensor Data Fusion System and its application” 2021 IEEE (Year: 2021). [cited by applicant]
Peli et al., “Feature Level Sensor Fusion” Part of the SPIE Conference on Sensor Fusion: Architectures. Algorithms, and Applications III. Orlando, Florida, Apr. 1999 (Year: 1999). [cited by applicant]
Cited By (4)
US 12,535,779 US 12,664,236 US 12,682,017 US 12,688,260