IP Library › Granted Patent US 12,270,628
Granted Patent B1
US 12,270,628 · App. 18/474,817 · Granted Apr 8, 2025

Time synchronization of optics using power feeds

Inventors: Frank R Stolle (Amherst, MA); Simone B. Bortolami (Belmont, MA); Larry H Mack (Pflugerville, TX)
Assignee: BAE Systems Information and Electronic Systems Integration Inc.
F41G3/065F41G3/08G06G7/80
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,270,628
App. No.
18/474,817
Granted
Apr 8, 2025
Kind
B1
Abstract

A weapon-mountable smart optic comprising: a time reference configured to output a signal comprising a periodically-repeating feature and time metadata and comprising a first oscillator; at least two sensors configured to gather data, each comprising secondary oscillators; and at least one processor in communication with each of the at least two sensors; wherein each of the at least two sensors is in operative communication with the time reference and is configured to associate an edge of the periodically-repeating signal with a time conveyed by the time metadata, and wherein each of the at least two sensors is configured to gather data, associate time metadata with the gathered data, and to send the gathered data with time metadata to the at least one processor, and wherein the at least one processor is configured to fuse the data gathered by each of the at least two sensors.

Claims (35)

1. A smart optic, the smart optic comprising:

a mount configured to allow at least a portion of the smart optic to be mounted to a weapon;

a time reference comprising a first oscillator, wherein the time reference is configured to output a signal comprising a periodically-repeating feature and time metadata;

at least two sensors configured to gather data, each of the at least two sensors comprising secondary oscillators; and

at least one processor in operative communication with at least one non-transitory storage medium and each of the at least two sensors;

wherein each of the at least two sensors is in operative communication with the time reference and is configured to associate an edge of the periodically-repeating signal with a time conveyed by the time metadata, and

wherein each of the at least two sensors is configured to gather data, associate time metadata with the gathered data, and to send the gathered data with time metadata to the at least one processor, and

wherein the at least one processor is configured to fuse the data gathered by each of the at least two sensors.

2. The smart optic of claim 1 , wherein the time metadata comprises an absolute time.

3. The smart optic of claim 1 , wherein the time reference comprises a GPS or GNSS disciplined oscillator.

4. The smart optic of claim 1 , wherein the time reference comprises an atomic clock.

5. The smart optic of claim 1 , wherein the time reference comprises a temperature compensated crystal oscillator.

6. The smart optic of claim 1 , wherein the time reference is internal to the smart optic.

7. The smart optic of claim 1 , wherein the signal is overlaid on a power feed to the smart optic and passes through the time reference.

8. The smart optic of claim 7 , wherein the signal overlaid on the power feed comprises an embedded voltage spike, or pulse, at a predetermined cadence.

9. The smart optic of claim 8 , wherein the pulse comprises a pulse train or specific waveform.

10. The smart optic of claim 7 , wherein a cadence of the pulse is one pulse per second.

11. The smart optic of claim 7 , wherein the power feed is provided by a rail system used to mount the smart optic to a weapon.

12. The smart optic of claim 1 , wherein the time reference is external to the smart optic.

13. The smart optic of claim 1 , further comprising one or more additional time references, with at least one of the additional time references being associated with at least one sensor.

14. The smart optic of claim 13 , wherein the at least one additional time references are synchronized to the time reference.

15. The smart optic of claim 1 , wherein one of the at least two sensors is an attitude sensor and wherein the attitude sensor is selected from the group consisting of gyroscopes, inertial measurement units, and multi-antenna GPS or GNSS modules.

16. A system of synchronized smart optics, the system comprising:

a first smart optic in accordance with claim 1 ,

at least one additional smart optic in accordance with claim 1 ,

wherein the first and additional smart optic are in operative communication with one another, and

wherein one of the first and additional smart optic is configured to synchronize its time reference to the time reference of other.

17. The system of claim 16 , wherein data generated by the first smart optic is used to correct for errors on the additional second smart optic.

18. The system of claim 16 wherein time references used on the first and additional smart optic are disciplined to one another prior to use.

19. The system of claim 16 wherein the first and additional smart optic are placed into master/slave relationships, with the time reference of a master smart optic being used to discipline oscillators in slave smart optics.

20. A system of synchronized smart optics, the system comprising:

at least two smart optics in accordance with claim 1 ; and

a synchronization module disposed between the at least two smart optics,

wherein the smart optics are in operative communication with the synchronization module, and

wherein the synchronization module is configured to synchronize the time references of the smart optics.

Assignments (2)
CONFIRMATORY LICENSE Recorded Feb 24, 2025
From: BAE SYSTEMS I&ES INTEGRATION INC
To: US GOVERNMENT AS REPRESENTED BY THE SECRETARY OF THE ARMY
Reel/Frame 070301/0120 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2023
From: STOLLE, FRANK R; BORTOLAMI, SIMONE B; MACK, LARRY H
To: BAE SYSTEMS INFORMATION AND ELECTRONIC SYSTEMS INTEGRATION INC.
Reel/Frame 065899/0449 →
References Cited (54)
US 9766074B2 · Roumeliotis · 2017 [cited by applicant]
US 10012504B2 · Roumeliotis · 2018 [cited by applicant]
US 10107919B1 · Chapman · 2018 [cited by examiner]
US 10151588B1 · Singh · 2018 [cited by applicant]
US 10254118B2 · Roumeliotis · 2019 [cited by applicant]
US 10325411B1 · Laney · 2019 [cited by applicant]
US 10366549B1 · Mash · 2019 [cited by applicant]
US 10694148B1 · Li · 2020 [cited by applicant]
US 10907971B2 · Roumeliotis · 2021 [cited by applicant]
US 10929713B2 · Chiu · 2021 [cited by applicant]
US 11466990B2 · Roumeliotis · 2022 [cited by applicant]
US 11544161B1 · Yarlagadda · 2023 [cited by applicant]
US 20070070069A1 · Samarasekera · 2007 [cited by applicant]
US 20110218733A1 · Hamza · 2011 [cited by applicant]
US 20130314509A1 · Laine · 2013 [cited by applicant]
US 20150286217A1 · Qian · 2015 [cited by applicant]
US 20170329335A1 · Delmarco · 2017 [cited by applicant]
US 20180091746A1 · Benser · 2018 [cited by applicant]
US 20190051056A1 · Chiu · 2019 [cited by applicant]
US 20190368877A1 · O'Shea · 2019 [cited by applicant]
US 20210048821A1 · Bondurant · 2021 [cited by applicant]
US 20220065571A1 · Canty · 2022 [cited by examiner]
WO WO2016174659A1 · 2016 [cited by examiner]
DelMarco, S., Webb, H., and Tom, V., “Spectrally-Shaped Correlation with Application to Image Registration,” Proceedings of SPIE, vol. 10993, 2019. [cited by applicant]
DelMarco, S., Webb, H., and Tom, V., “A Progressive Refinement Approach to Aerial Image Registration Using Local Transform Perturbations,” Proceedings of the IEEE International Geoscience and Remote Sensing. Symposium, … [cited by applicant]
DelMarco, S., “Multi-Template Image Matching Using Alpha-Rooted Biquaternion Phase Correlation with Application to Logo Recognition,” Proceedings of SPIE, vol. 8063, 2011. [cited by applicant]
DelMarco, S., “Multiple Template-Based Image Matching Using Alpha-Rooted Quaternion Phase Correlation,” Proceedings of SPIE, vol. 7708, 2010. [cited by applicant]
DelMarco, S., “Logo Recognition Using Alpha-Rooted Phase Correlation in the Radon Transform Domain,” Proceedings of SPIE, vol. 7443, 2009. [cited by applicant]
DelMarco, S., Tom, V., Webb, H., and Lefebvre, D., “A Verification Metric for Multi-Sensor Image Registration,” Proceedings of SPIE, vol. 6567, 2007. [cited by applicant]
DelMarco, S., Webb, H., and Tom, V., “Automatic Spatial Accuracy Estimation for Correlation-Based Image Registration,” Proceedings of SPIE, vol. 10993, 2019. [cited by applicant]
Naman Patel et al. “Sensor Modality Fusion with CNNs for UGV Autonomous Driving Indoor Environments” 8 pages. [cited by applicant]
Shahram Moafipoor et al. “LiDAR/Camera Point & Pixel Aided Autonomous Navigation” LiDAR Magazine, vol. 7 No. 7, 7 pages, https://lidarmag.com/2017/10/25/lidarcamera-point-pixel-aided-autonomous-navigation/. [cited by applicant]
Qi Kong et al. “Outdoor real-time RGBD sensor fusion of stereo camera and sparse lidar” (2022) J. Phys.: Conf. Ser. 2234 012010, doi:10.1088/1742-6596/2234/1/012010. [cited by applicant]
Darshan Bhanushali et al. “LiDAR-Camera Fusion for 3D Object Detection” in Proc. IS&T Int'l. Symp. on Electronic Imaging: Autonomous Vehicles and Machines, 2020, pp. 257-1-257-9, https://doi.org/10.2352/ISSN.2470-1173.2… [cited by applicant]
Yilin Zhang “Lidar—camera deep fusion for end-to-end trajectory planning autonomous vehicle” 2022 J. Phys.: Conf. Ser. 2284 012006, retrieved from the internet: https://iopscience.iop.org/article/10.1088/1742-6596/2284/… [cited by applicant]
Anderson Lebbad, C. Nataraj “A Bayesian Alogorithm for Vision Based Navigation of Autonomous Surface Vehicles” 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on R… [cited by applicant]
“A Novel Approach for Fusing LIDAR and Visual Camera images in Unstructured environment” International Conference on Advanced Computing and Communication Systems (ICACCS-2017), Jan. 6-7, 2017, Coimbatore, India. [cited by applicant]
Qingquan Li et al. “A Sensor-Fusion Drivable-Region and Lane-Detection System for Autonomous Vehicle Navigation in Challenging Road Scenarios” IEEE Transactions On Vehicular Technology, vol. 63, No. 2, p. 540-555, Feb. … [cited by applicant]
“Sebastion Schneider et al. ““Fusing Vision and LIDAR—Synchronization, Correction and Occlusion Reasoning”” 2010 IEEE Intelligent Vehicles SymposiumUniversity of California, San Diego, CA, USA Jun. 21-24, 2010, 6 pages”. [cited by applicant]
Xiangmo Zhao et al. “Fusion of 3D LIDAR and Camera Data for Object Detection in Autonomous Vehicle Applications” IEEE Sensors Journal, vol. 20, No. 9, May 1, 2020, 13 pages. [cited by applicant]
Hyunggi Jo et al. “New Monte Carlo Localization Using Deep Initialization: A Three-Dimensional LiDAR and a Camera Fusion Approach” vol. 8 2020, School of Electrical and Electronic Engineering, Yonsei University, Seoul 0… [cited by applicant]
Qingqu Wang et al. “On SLAM Based on Monocular Vision and Lidar Fusion System” 2018 IEEE Csaa Guidance, Navigation and Control Conference (CGNCC), Xiamen, China, 2018, pp. 1-5, doi: 10.1109/GNCC42960.2018.9018987. [cited by applicant]
Jing Li et al. “OpenStreetMap-Based Autonomous Navigation for the Four Wheel-Legged Robot Via 3D-Lidar and CCD Camera” in IEEE Transactions on Industrial Electronics, vol. 69, No. 3, pp. 2708-2717, Mar. 2022, doi: 10.11… [cited by applicant]
Mao Shan et al “Probabilistic Egocentric Motion Correction of Lidar Point Cloud and Projection to Camera Images for Moving Platforms” 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC),… [cited by applicant]
Cesar Debeunne et al. “A Review of Visual-LiDAR Fusion based Simultaneous Localization and Mapping” Published Apr. 7, 2020, retrieved from the internet www.mdpi.com/journal/sensors, Sensors 2020, 20, 2068; doi:10.3390/s… [cited by applicant]
Shi-Sheng Huang et al. “Lidar-Monocular Visual Odometry using Point and Line Features” 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, 2020, pp. 1091-1097, doi: 10.1109/ICRA40945.202… [cited by applicant]
Johannes Graeter et al. “LIMO: Lidar-Monocular Visual Odometry” 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 2018, pp. 7872-7879, doi: 10.1109/IROS.2018.8594394. [cited by applicant]
Young-Sik Shin et al. “Direct Visual SLAM using Sparse Depth for Camear-LiDAR System” 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia, 2018, pp. 5144-5151, doi: 10.1109/ICR… [cited by applicant]
Ben Mildenhall et al. “NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis” 17 pages. [cited by applicant]
Stephan J. Garbin et al. “FastNeRF: High-Fidelity Neural Rendering at 200FPS” 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 2021, pp. 14326-14335, doi: 10.1109/ICCV48922.2021.01… [cited by applicant]
Sara Fridovich-Keil et al. “Plenoxels: Radiance Fields without Neural Networks” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 5491-5500, doi: 10.1109/CVPR526… [cited by applicant]
“Stephen DelMarco et al. ““High-Precision Infantry Training System (HITS)”” Proceedings of the 33rd International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2020), Sep. 2020, p… [cited by applicant]
“Stephen DelMarco, Helen Webb, Victor Tom ““Automatic Spatial Accuracy Estimation Correlation-based Image Registration”” Proc. SPIE 10993, Mobile Multimedia/Image Processing, Security, and Applications 2019,1099307 (May… [cited by applicant]
“Stephen DelMarco, Helen Webb, Victor Tom ““Spectrally-shaped Correlation With Application to Image Registration”” Proc. SPIE 10993, Mobile Multimedia/Image Processing, Security, and Applications 2019, 1099306 (May 13, … [cited by applicant]