IP Library Granted Patent US 12,299,083
Granted Patent B1
US 12,299,083 · App. 19/016,267 · Granted May 13, 2025

Systems and methods of sensor data fusion

Inventor: Armando Montalvo (Winter Garden, FL)
Assignee: Digital Global Systems, Inc.
G06F18/251G06N5/046
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,299,083
App. No.
19/016,267
Filed
Jan 10, 2025
Granted
May 13, 2025
Kind
B1
Art Unit
2634
USPC
706/45
Abstract

Systems and methods of sensor data fusion including sensor data capture, curation, linking, fusion, inference, and validation. The systems and methods described herein reduce computational demand and processing time by curating data and calculating conditional entropy. The system is operable to fuse data from a plurality of sensor types. A computer processor optionally stores fused sensor data that the system validates above a mathematical threshold.

Claims (53)

1. A system for sensor data fusion for sensor management and utilization in robotics, comprising:

at least one computer processor including a memory;

at least one curation engine, at least one link engine, at least one fusion engine, at least one inference engine, and at least one validation engine;

at least one first position sensor operable to capture a first position measurement of a robotic component; and

at least one second position sensor operable to capture a second position measurement of the robotic component;

wherein the at least one computer processor is operable to analyze the first position measurement and the second position measurement;

wherein the at least one computer processor is operable to receive at least one query;

wherein the at least one curation engine is operable to curate the first position measurement and the second position measurement, the at least one link engine is operable to link the first position measurement and the second position measurement, the at least one fusion engine is operable to fuse the first position measurement and the second position measurement, the at least one inference engine is operable to determine at least one inference from the first position measurement and the second position measurement, and the at least one validation engine is operable to validate the first position measurement and the second position measurement;

wherein the at least one inference engine is operable to determine a second inference;

wherein the at least one validation engine is operable to use artificial intelligence to compare the at least one inference to the second inference;

wherein the at least one validation engine validates the at least one inference when the comparison between the at least one inference and the second inference exceeds a predefined threshold; and

wherein the at least one computer processor is operable to instruct the robotic component to move based on the at least one inference being validated.

2. The system of claim 1 , wherein the predefined threshold includes the at least one inference being within about 4.5% or less of the second inference.

3. The system of claim 1 , wherein the system is operable to store the first position measurement and the second position measurement after the at least one validation engine validates the at least one inference.

4. The system of claim 1 , wherein the at least one validation engine is operable to validate the at least one inference passively and/or actively.

5. The system of claim 4 , wherein passive validation includes not modifying a movement of the robotic component.

6. The system of claim 4 , wherein active validation includes modifying at least one movement of the robotic component.

7. The system of claim 1 , wherein the at least one validation engine is operable to use artificial intelligence to dynamically adjust the predefined threshold based in part on types of data sources, environmental factors, and/or learning from previous analyses conducted by the at least one validation engine.

8. A method for sensor data fusion for sensor management and utilization in robotics, comprising:

providing at least one computer processor including a memory;

providing at least one curation engine, at least one link engine, at least one fusion engine, at least one inference engine, and at least one validation engine;

at least one first position sensor capturing a first position measurement of a robotic component;

at least one second position sensor capturing a second position measurement of the robotic component;

analyzing by the at least one computer processor the first position measurement and the second position measurement;

receiving by the at least one computer processor at least one query;

curating by the at least one curation engine the first position measurement and the second position measurement, linking by the at least one link engine the first position measurement and the second position measurement, fusing by the at least one fusion engine the first position measurement and the second position measurement, determining at least one inference by the at least one inference engine from the first position measurement and the second position measurement, and validating by the at least one validation engine the first position measurement and the second position measurement;

determining by the at least one validation engine a second inference;

comparing by the at least one validation engine via artificial intelligence the at least one inference to the second inference;

validating by the at least one validation engine the at least one inference when the comparison between the at least one inference and the second inference exceeds a predefined threshold; and

instructing by the at least one computer processor the robotic component to move based on the at least one inference being validated.

9. The method of claim 8 , wherein the predefined threshold includes the at least one inference being within about 4.5% or less of the second inference.

10. The method of claim 8 , further comprising validating the at least one inference passively and/or actively.

11. The method of claim 10 , wherein validating passively includes not modifying a movement of the robotic component.

12. The method of claim 10 , wherein validating actively includes modifying a movement of the robotic component.

13. The method of claim 8 , further comprising adjusting via the at least one validation engine using artificial intelligence the predefined threshold based in part on types of data sources, environmental factors, and/or learning from previous analyses conducted by the at least one validation engine.

14. The method of claim 8 , further comprising storing the first position measurement and the second position measurement after the at least one validation engine validates the at least one inference.

15. A system for sensor data fusion for sensor management and utilization in robotics, comprising:

at least one computer processor including a memory;

at least one curation engine, at least one link engine, at least one fusion engine, at least one inference engine, and at least one validation engine; and

at least two sensors, each of the at least two sensors operable to measure a first position of a robotic component and a second position of the robotic component;

wherein the at least one computer processor is operable to analyze the first position and the second position;

wherein the at least one computer processor is operable to receive at least one query;

wherein the at least one curation engine is operable to curate the first position and the second position, the at least one link engine is operable to link the first position and the second position, the at least one fusion engine is operable to fuse the first position and the second position, the at least one inference engine is operable to determine at least one inference from the first position and the second position, and the at least one validation engine is operable to validate the first position and the second position;

wherein the at least one inference engine is operable to determine a second inference;

wherein the at least one validation engine is operable to use artificial intelligence to compare the at least one inference to the second inference;

wherein the at least one validation engine validates the at least one inference when the comparison between the at least one inference and the second inference exceeds a predefined threshold;

wherein the at least one validation engine is operable to use artificial intelligence to dynamically adjust the predefined threshold based in part on types of data sources, environmental factors, and/or learning from previous analyses conducted by the at least one validation engine; and

wherein the at least one computer processor is operable to instruct the robotic component to move based on the at least one inference being validated.

16. The system of claim 15 , wherein the predefined threshold includes the at least one inference being within about 4.5% or less of the second inference.

17. The system of claim 15 , wherein the at least one validation engine is operable to validate the at least one inference passively and/or actively.

18. The system of claim 17 , wherein passive validation includes not modifying a parameter.

19. The system of claim 17 , wherein active validation includes modifying at least one parameter.

20. The system of claim 15 , wherein the system is operable to store the first position and the second position after the at least one validation engine validates the at least one inference.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 13, 2025
From: MONTALVO, ARMANDO
To: DIGITAL GLOBAL SYSTEMS, INC.
Reel/Frame 070214/0056 →
Continuity (2)
Continuation 18990248 · Dec 20, 2024
Continuation 18988120 · Dec 19, 2024
References Cited (44)
US 9367683B2 · Kolacinski et al. · 2016 [cited by applicant]
US 10748075B2 · Chowdhary et al. · 2020 [cited by applicant]
US 10766137B1 · Porter · 2020 [cited by examiner]
US 11037320B1 · Ebrahimi Afrouzi et al. · 2021 [cited by applicant]
US 11586865B2 · Brahma et al. · 2023 [cited by applicant]
US 20080312756A1 · Grichnik et al. · 2008 [cited by applicant]
US 20150306770A1 · Mittal · 2015 [cited by examiner]
US 20170004411A1 · Hassan · 2017 [cited by examiner]
US 20180284758A1 · Cella et al. · 2018 [cited by applicant]
US 20190295001A1 · Fusco · 2019 [cited by examiner]
US 20200026289A1 · Alvarez et al. · 2020 [cited by applicant]
US 20210150230A1 · Smolyanskiy et al. · 2021 [cited by applicant]
US 20210342656A1 · Mittal et al. · 2021 [cited by applicant]
US 20220012562A1 · Lazaro-Gredilla · 2022 [cited by examiner]
US 20220075077A1 · Kato et al. · 2022 [cited by applicant]
US 20220126864A1 · Moustafa et al. · 2022 [cited by applicant]
US 20220146652A1 · Cardei et al. · 2022 [cited by applicant]
US 20220161815A1 · Van Beek et al. · 2022 [cited by applicant]
US 20220179056A1 · Braley et al. · 2022 [cited by applicant]
US 20230110027A1 · Bajpayee et al. · 2023 [cited by applicant]
US 20230112441A1 · Tang et al. · 2023 [cited by applicant]
US 20230166765A1 · Yoon et al. · 2023 [cited by applicant]
US 20230342944A1 · Sullivan et al. · 2023 [cited by applicant]
US 20240045426A1 · Ditty et al. · 2024 [cited by applicant]
US 20240112428A1 · Levi et al. · 2024 [cited by applicant]
US 20240142994A1 · Ebrahimi Afrouzi · 2024 [cited by applicant]
US 20240152734A1 · Ye · 2024 [cited by examiner]
US 20240249165A1 · Roche · 2024 [cited by examiner]
US 20240289930A1 · Yang et al. · 2024 [cited by applicant]
US 20240289981A1 · Kuo et al. · 2024 [cited by applicant]
US 20240296044A1 · Day et al. · 2024 [cited by applicant]
US 20240312219A1 · Choi et al. · 2024 [cited by applicant]
US 20240317254A1 · Tran · 2024 [cited by applicant]
US 20240324838A1 · Ebrahimi Afrouzi · 2024 [cited by applicant]
US 20240331403A1 · Shen et al. · 2024 [cited by applicant]
US 20240378412A1 · Schrader · 2024 [cited by applicant]
US 20240408495A1 · Hibi et al. · 2024 [cited by applicant]
CN 110598299A · 2019 [cited by applicant]
CN 116522279A · 2023 [cited by applicant]
Chen et al., “Track correlation uncertainty evaluation for Multi-sensor Data Fusion System and its application” 2021 IEEE (Year: 2021). [cited by applicant]
Peli et al., “Feature Level Sensor Fusion” Part of the SPIE Conference on Sensor Fusion: Architectures. Algorithms, and Applications III. Orlando, Florida, Apr. 1999 (Year: 1999). [cited by applicant]
Ebel, Patrick, and et al. “SEN12MS-CR-TS: A remote-sensing data set for multi modal multitemporal cloud removal.” IEEE Transactions on Geoscience and Remote Sensing 60 (2022): 1-14 (Year: 2022). [cited by applicant]
Wei Z, and et al. MmWave Radar and Vision Fusion for Object Detection in Autonomous Driving: A Review. Sensors (Basel). Mar. 25, 2022;22(7):2542. doi: 10.3390/s22072542. PMID: 35408157; PMCID: PMC9003130 (Year: 2022). [cited by applicant]
Yeong, D. J., Velasco-Hernandez, G., Barry, J., & Walsh, J. (2021). Sensor and sensor fusion technology in autonomous vehicles: A review. Sensors, 21(6), 2140. (Year: 2021). [cited by applicant]
Cited By (3)
US 12,664,236 US 12,682,017 US 12,688,260