IP Library › Granted Patent US 12,375,791
Granted Patent B2
US 12,375,791 · App. 18/367,622 · Granted Jul 29, 2025

System for underwater depth perception having multiple image sensors

Inventors: Christopher Allan Gilson (Markham, CA); Stanley James Brown (Waterloo, CA)
Assignee: VOYIS IMAGING INC.
H04N23/555H04N13/239H04N23/54H04N23/90H04N2013/0081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,375,791
App. No.
18/367,622
Granted
Jul 29, 2025
Kind
B2
Abstract

Systems described herein use either multiple image sensors or one image sensor and a complementary sensor to provide underwater depth perception. The systems include a computing system that provides the underwater depth perception based on data sensed by the multiple image sensors or the one image sensor and the complementary sensor. The systems can include a submersible device (such as a submersible mobile machine) that includes a holder configured to hold the one image sensor. The holder can be configured to hold the computing system in addition to the one image sensor. And, in some embodiments, the holder is configured to hold the complementary sensor in addition to the computing system and the one image sensor. Alternatively, in some embodiments, the holder is configured to hold the multiple image sensors. And, the holder can be configured to hold the computing system in addition to the multiple image sensors.

Claims (72)

1. A system, comprising:

image sensors, arranged on different axes and configured to complement each other, and each one of the image sensors comprising any type of sensor that captures an image based on reflected or emitted electromagnetic radiation or mechanical waves;

a computing system, configured to provide underwater depth perception based on data captured by the image sensors; and

a submersible device, comprising a holder, the holder configured to hold the image sensors and the computing system, and wherein computing by the computing system to provide the underwater depth perception occurs onboard the submersible device,

wherein the computing system and the image sensors are configured to operate together to measure a relative position of an object in images captured by the image sensors to provide the underwater depth perception,

wherein the computing system is configured to select a feature between two image sensors of the image sensors to calculate a distance between the selected feature and the object and to provide the underwater depth perception,

wherein the computing system is configured to perform the measurement of the relative position of the object in the images using triangulation,

wherein the computing system is configured to perform the triangulation, and wherein the triangulation is based on respective observations of the object by the two image sensors, a known distance between the two image sensors that defines a base, and respective angles between the base and respective rays corresponding to the respective observations of the object,

wherein the two sensors are configured to observe the object according to respective rays beginning at respective starting points at the two sensors and extending towards a selected point on a surface of the object,

wherein the computing system is configured to use the respective starting points of the rays and the selected point on the surface of the object to define a spatial triangle for the triangulation,

wherein the computing system is configured to:

use the known distance between the two sensors as a base of the spatial triangle;

determine angles between the base and the respective rays corresponding to the respective observations of the object;

use the determined angles to determine the intersection point of the respective rays to provide a spatial coordinate of the selected point on the surface of the object according to triangular relations; and

use triangular relations to determine the distance between the selected feature and the point on the surface of the object and to provide the underwater depth perception,

wherein the computing system is configured to generate an underwater three-dimensional model based on the data captured by the respective image sensors,

wherein the model comprises a plurality of distances between the selected feature and objects underwater, and

wherein the plurality of distances between the selected feature and the objects underwater comprise the determined distance between the selected feature and the point on the surface of the object.

2. The system of claim 1 , wherein the image sensors consist of a pair of sensors that sense reflected electromagnetic energy.

3. The system of claim 1 , further comprising a stereo camera that comprises the image sensors and that is configured to capture three-dimensional images using the image sensors.

4. The system of claim 3 , wherein the image sensors consist of a pair of image sensors of the camera.

5. The system of claim 1 , wherein the submersible device comprises a submersible mobile machine.

6. The system of claim 5 , wherein the submersible mobile machine comprises a submersible robot.

7. The system of claim 5 , wherein the submersible mobile machine comprises a submersible vehicle.

8. The system of claim 7 , wherein the submersible vehicle is a submarine or a submersible.

9. The system of claim 8 , wherein the image sensors and the computing system are configured to operate effectively while the submersible device and the holder are completely submerged.

10. The system of claim 1 , wherein the computing system is further configured to provide underwater depth perception based on raw data captured by the image sensors.

11. A system, comprising:

image sensors, arranged on different axes and configured to complement each other, and each one of the image sensors comprising any type of sensor that captures an image based on reflected or emitted electromagnetic radiation or mechanical waves;

a computing system, configured to provide underwater depth perception based on data captured by the image sensors; and

a submersible device, comprising a holder, the holder configured to hold the image sensors and the computing system, and wherein computing by the computing system to provide the underwater depth perception occurs onboard the submersible device,

wherein the computing system and the image sensors are configured to operate together to measure a relative position of an object in images captured by the image sensors to provide the underwater depth perception,

wherein the computing system is configured to select a feature between two image sensors of the image sensors to calculate a distance between the selected feature and the object and to provide the underwater depth perception,

wherein the computing system is configured to perform the measurement of the relative position of the object in the images using triangulation,

wherein the computing system is configured to perform the triangulation, and wherein the triangulation is based on respective observations of the object by the two image sensors, a known distance between the two image sensors that defines a base, and respective angles between the base and respective rays corresponding to the respective observations of the object,

wherein the two sensors are configured to observe the object according to respective rays beginning at respective starting points at the two sensors and extending towards a selected point on a surface of the object,

wherein the computing system is configured to use the respective starting points of the rays and the selected point on the surface of the object to define a spatial triangle for the triangulation,

wherein the computing system is configured to:

use the known distance between the two sensors as a base of the spatial triangle;

determine angles between the base and the respective rays corresponding to the respective observations of the object;

use the determined angles to determine the intersection point of the respective rays to provide a spatial coordinate of the selected point on the surface of the object according to triangular relations; and

use triangular relations to determine the distance between the selected feature and the point on the surface of the object and to provide the underwater depth perception,

wherein the computing system is configured to generate an underwater depth map based on the data captured by the respective image sensors,

wherein the map comprises a plurality of distances between the selected feature and objects underwater, and

wherein a plurality of distances between the selected feature and the objects underwater comprise the determined distance between the selected feature and the point on the surface of the object.

12. The system of claim 11 , wherein the image sensors consist of a pair of sensors that sense reflected electromagnetic energy.

13. The system of claim 11 , further comprising a stereo camera that comprises the image sensors and that is configured to capture three-dimensional images using the image sensors.

14. The system of claim 13 , wherein the image sensors consist of a pair of image sensors of the camera.

15. The system of claim 11 , wherein the submersible device comprises a submersible mobile machine.

16. The system of claim 15 , wherein the submersible mobile machine comprises a submersible robot.

17. The system of claim 15 , wherein the submersible mobile machine comprises a submersible vehicle.

18. The system of claim 17 , wherein the submersible vehicle is a submarine or a submersible.

19. A system, comprising:

image sensors, arranged on different axes and configured to complement each other, and each one of the image sensors comprising any type of sensor that captures an image based on reflected or emitted electromagnetic radiation or mechanical waves;

a computing system, configured to provide underwater depth perception based on data captured by the image sensors; and

a submersible device, comprising a holder, the holder configured to hold the image sensors and the computing system, and wherein computing by the computing system to provide the underwater depth perception occurs onboard the submersible device,

wherein the computing system and the image sensors are configured to operate together to measure a relative position of an object in images captured by the image sensors to provide the underwater depth perception,

wherein the computing system is configured to select a feature between two image sensors of the image sensors to calculate a distance between the selected feature and the object and to provide the underwater depth perception,

wherein the computing system is configured to perform the measurement of the relative position of the object in the images using triangulation,

wherein the computing system is configured to perform the triangulation, and wherein the triangulation is based on respective observations of the object by the two image sensors, a known distance between the two image sensors that defines a base, and respective angles between the base and respective rays corresponding to the respective observations of the object,

wherein the two sensors are configured to observe the object according to respective rays beginning at respective starting points at the two sensors and extending towards a selected point on a surface of the object,

wherein the computing system is configured to use the respective starting points of the rays and the selected point on the surface of the object to define a spatial triangle for the triangulation,

wherein the computing system is configured to:

use the known distance between the two sensors as a base of the spatial triangle;

determine angles between the base and the respective rays corresponding to the respective observations of the object;

use the determined angles to determine the intersection point of the respective rays to provide a spatial coordinate of the selected point on the surface of the object according to triangular relations; and

use triangular relations to determine the distance between the selected feature and the point on the surface of the object and to provide the underwater depth perception,

wherein the computing system is configured to generate an underwater depth map based on the data captured by the respective image sensors,

wherein the map comprises a plurality of distances between the selected feature and objects underwater,

wherein a plurality of distances between the selected feature and the objects underwater comprise the determined distance between the selected feature and the point on the surface of the object, and

wherein the submersible device comprises a mobile machine.

20. The system of claim 19 , wherein the mobile machine comprises a submersible robot or a submersible vehicle.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 5, 2023
From: BROWN, STANLEY JAMES; GILSON, CHRISTOPHER ALLAN
To: VOYIS IMAGING INC.
Reel/Frame 065138/0229 →
Continuity (1)
Related Publication 20250088726A1 · Mar 13, 2025
References Cited (50)
US 8542413B2 · Gillham et al. · 2013 [cited by applicant]
US 9612332B2 · Wicks et al. · 2017 [cited by applicant]
US 10116841B2 · Boyle et al. · 2018 [cited by applicant]
US 10116842B2 · Boyle et al. · 2018 [cited by applicant]
US 10158793B2 · Boyle et al. · 2018 [cited by applicant]
US 10163213B2 · Boyle et al. · 2018 [cited by applicant]
US 10930013B2 · Boyle et al. · 2021 [cited by applicant]
US 10943321B2 · Boyle et al. · 2021 [cited by applicant]
US 11457144B2 · Wallace et al. · 2022 [cited by applicant]
US 11561553B1 · Dydek · 2023 [cited by examiner]
US 20080319704A1 · Forster · 2008 [cited by examiner]
US 20140300885A1 · Debrunner et al. · 2014 [cited by applicant]
US 20170046845A1 · Boyle et al. · 2017 [cited by applicant]
US 20190011552A1 · Embry et al. · 2019 [cited by applicant]
US 20190017941A1 · Gillham et al. · 2019 [cited by applicant]
US 20210201452A1 · Treibitz · 2021 [cited by applicant]
US 20230045358A1 · Kobayashi · 2023 [cited by examiner]
AU 2015250746B2 · 2015 [cited by applicant]
Moore et. al., Development of a New Underwater Bathymetric Laser Imaging System: L-Bath, Aug. 1, 2000. [cited by applicant]
Nils Tornblom, Underwater 3D Surface Scanning using Structured Light, Dec. 2010. [cited by applicant]
Bruno et al., Experimentation of structured light and stereo vision for underwater 3D reconstruction, Jul. 2011. [cited by applicant]
Roman et al., Application of Structured Light Imaging for High Resolution Mapping of Underwater Archaeological Sites, Abstract, May 2010. [cited by applicant]
Jaffe et al., Underwater Optical Imaging: Status and Prospects, 2011. [cited by applicant]
Scharstein et al., High-Accuracy Stereo Depth Maps Using Structured Light, Jun. 2003. [cited by applicant]
Hall-Holt et al., Stripe Boundary Codes for Real-Time Structured-Light Range Scanning of Moving Objects, Abstract, 2001. [cited by applicant]
Caimi et al., Underwater Optics and Imaging, 2008. [cited by applicant]
Mullen et al., Modulated Laser Line Scanner for Enhanced Underwater Imaging, Abstract, Nov. 11, 1998. [cited by applicant]
Weiqi et al., Range-gated underwater laser imaging system based on intensified gate imaging technology, Abstract, Feb. 2008. [cited by applicant]
Fournier et al., LUCIE ROV mounted laser imaging system, Abstract, Dec. 1992. [cited by applicant]
Bonin et al., Imaging Systems for Advanced Underwater Vehicles, Feb. 10, 2011. [cited by applicant]
Alexander Rumson, The application of fully unmanned robotic systems for inspection of subsea pipelines, Sep. 1, 2021. [cited by applicant]
Maki et al., Development of the AUV Tri-TON, Abstract, 2013. [cited by applicant]
Brignone et al., First sea trials of a laser aided three dimensional underwater image mosaicing technique, 2011. [cited by applicant]
Nakatani et al., 3D Visual Modeling of Hydrothermal Chimneys using a Rotary Laser Scanning System, Abstract, 2011. [cited by applicant]
Maki et al., Volumetric mapping of tube worm colonies in Kagoshima Bay through autonomous robotic surveys, Jul. 2011. [cited by applicant]
Maki et al., Autonomous detection and vol. determination of tubeworm colonies from underwater robotic surveys, Abstract, 2010. [cited by applicant]
Maki et al., Real time path-planning of an AUV based on characteristics of passiveacoustic landmarks for visual mapping of shallow vent fields, Abstract, 2007. [cited by applicant]
Levoy et al., Improving underwater vision using confocal imaging, Jun. 28, 2009. [cited by applicant]
Vasilescu et al., Color-Accurate Underwater Imaging using Perceptual Adaptive Illumination, Aug. 2011. [cited by applicant]
Strand et al., Laser line-scan fluorescence and multispectral imaging of coral reef environments, Abstract, 1997. [cited by applicant]
Goddijn-Murphy et al., Fundamentals of in Situ Digital Camera Methodology for Water Quality Monitoring of Coast and Ocean, Jul. 22, 2009. [cited by applicant]
https://www.ivm-technologies.com/#1. [cited by applicant]
https://vaarst.com/subslam-3d-imaging-technology/. [cited by applicant]
https://zupt.com/services/3d-recon-20. [cited by applicant]
https://cathxocean.com/vanguard/. [cited by applicant]
https://www.savante.co.uk/subsea-cameras-products/vergence-hd-stereo-photogrammetry-camera. [cited by applicant]
Akkaynak et al., Sea-Thur: A Method for Removing Water From Underwater Images, Abstract, 2019. [cited by applicant]
International Search Report for PCT International Application Serial No. PCT/CA2024/051128. [cited by applicant]
Written Opinion of the International Search Authority for PCT International Application Serial No. PCT/CA2024/051128. [cited by applicant]
Computer stereo vision, from Wikipedia of Apr. 22, 2022, https://web.archive.org/web/20220422153444/https://en.wikipedia.org/wiki/Computer_stereo_vision. [cited by applicant]
Cited By (1)
US 12,475,581