IP Library Granted Patent US 12,193,418
Granted Patent B2
US 12,193,418 · App. 17/706,862 · Granted Jan 14, 2025

Sensor positioning system

Inventors: Matthew Messana (Sunnyvale, CA); Kyle James Cormany (Atascadero, CA); Christopher Thornton (Vancouver, CA); Barnaby John James (Campbell, CA); Neil Davé (San Mateo, CA); Shane Washburn (Oakland, CA)
Assignee: TidalX AI Inc.
A01K61/10A01K61/65A01K61/85B25J13/085B25J13/087B25J13/089B25J19/021G03B17/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,193,418
App. No.
17/706,862
Granted
Jan 14, 2025
Kind
B2
Abstract

A sensor positioning system, includes an actuation server for communicating with components of the sensor positioning system. The sensor positioning system additionally includes a first actuation system and a second actuation system, wherein each actuation system includes a pulley system for maneuvering an underwater sensor system. The sensor positioning system includes a dual point attachment bracket that connects through a first line to the first actuation system and connecting through a second line to the second actuation system. The underwater sensor system is affixed to the first pulley system, the second pulley system, and the dual attachment bracket through the first line and the second line.

Claims (36)

1. A method performed by one or more processing devices, the method comprising:

receiving, by the one or more processing devices, data indicating an activity to be performed by a movable underwater sensor system in an aquatic structure;

obtaining, by the one or more processing devices, (i) winch commands for moving the underwater sensor system into a position in the aquatic structure to perform the activity and (ii) instructions for the underwater sensor system to perform a measurement associated with the activity;

transmitting, by the one or more processing devices, the winch commands to a winch to move one or more lines coupled to the underwater sensor system to maneuver the underwater sensor system to the position to perform the activity; and

after the underwater sensor system has reached the position to perform the activity, transmitting, by the one or more processing devices, the instructions for the underwater sensor system to perform the measurement associated with the activity.

2. The method of claim 1 , wherein the one or more processing devices are configured to adjust the position of the underwater sensor system using closed-loop feedback.

3. The method of claim 1 , wherein the position for the underwater sensor system to perform the activity is based on output of a machine learning model.

4. The method of claim 1 , wherein obtaining the winch commands comprises verifying that the winch command can be validly executed based on parameters of the movable underwater sensor system.

5. The method of claim 1 , wherein obtaining the winch commands comprises determining depth measurements based on input from an absolute pressure sensor, a sonar sensor, a laser range finder, a water temperature sensor, or an ambient light level sensor.

6. The method of claim 1 , wherein obtaining the winch commands comprises determining distance measurements with respect to an element of the aquatic structure based on input from a sonar sensor, a laser range finder, or 3-D reconstruction from images from a stereo camera system.

7. The method of claim 1 , wherein obtaining the winch commands comprises determining line tension measurements based on input from a load cell, a motor torque sensor, a motor current sensor.

8. The method of claim 1 , wherein obtaining the winch commands comprises determining line length estimates based on (i) a rotational position of motors determined using an encoder, resolver, or hall effect sensor, (ii) an angular position sensor, or (iii) a mechanism for measuring active diameter of spools as line is fed in and out.

9. The method of claim 1 , wherein obtaining the winch commands comprises obtaining commands for each of at least two motorized pulley systems that independently maneuver one or more lines that are coupled to the underwater sensor system.

10. The method of claim 1 , wherein the instructions for the underwater sensor system include instructions to operate a pan motor atop the underwater sensor system.

11. The method of claim 1 , wherein the activity is automatically determined based on images that were previously generated by the underwater sensor system.

12. The method of claim 1 , wherein the underwater sensor system comprises a stereo camera.

13. The method of claim 1 , wherein the activity is automatically determined based on a pre-programmed schedule of activities that are to be performed by the underwater sensor system.

14. The method of claim 1 , wherein the winch commands specify coordinates within the aquatic structure.

15. The method of claim 1 , wherein the winch commands specify rotational amounts.

16. The method of claim 1 , wherein the winch commands specify a series of waypoints and times associated with each waypoint.

17. The method of claim 1 , comprising:

detecting a fault; and

transmitting additional winch commands the winch to move the one or more lines coupled to the underwater sensor system to maneuver the underwater sensor system to a predetermined location.

18. The method of claim 1 , wherein the winch commands are obtained based at least in part on current position data sent by the winch to the one or more processing devices.

19. A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving data indicating an activity to be performed by a movable underwater sensor system in an aquatic structure;

obtaining (i) winch commands for moving the underwater sensor system into a position in the aquatic structure to perform the activity and (ii) instructions for the underwater sensor system to perform a measurement associated with the activity;

transmitting the winch commands to a winch to move one or more lines coupled to the underwater sensor system to maneuver the underwater sensor system to the position to perform the activity; and

after the underwater sensor system has reached the position to perform the activity, transmitting the instructions for the underwater sensor system to perform the measurement associated with the activity.

20. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving data indicating an activity to be performed by a movable underwater sensor system in an aquatic structure;

obtaining (i) winch commands for moving the underwater sensor system into a position in the aquatic structure to perform the activity and (ii) instructions for the underwater sensor system to perform a measurement associated with the activity;

transmitting the winch commands to a winch to move one or more lines coupled to the underwater sensor system to maneuver the underwater sensor system to the position to perform the activity; and

after the underwater sensor system has reached the position to perform the activity, transmitting the instructions for the underwater sensor system to perform the measurement associated with the activity.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2024
From: X DEVELOPMENT LLC
To: TIDALX AI INC.
Reel/Frame 068477/0306 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 29, 2022
From: MESSANA, MATTHEW; CORMANY, KYLE JAMES; THORNTON, CHRISTOPHER; JAMES, BARNABY JOHN; DAVÉ, NEIL; WASHBURN, SHANE
To: X DEVELOPMENT LLC
Reel/Frame 059423/0184 →
Continuity (3)
Division 16385292 · Apr 16, 2019
Provisional Application 62742145 · Oct 5, 2018
Related Publication 20220217952A1 · Jul 14, 2022
References Cited (123)
US 4495891A · Dugan et al. · 1985 [cited by applicant]
US 6262761B1 · Zernov et al. · 2001 [cited by applicant]
US 10064396B2 · Lyngoy · 2018 [cited by applicant]
US 10856520B1 · Kozachenok et al. · 2020 [cited by applicant]
US 11266128B2 · Yao · 2022 [cited by examiner]
US 20030000653A1 · Ulatowski et al. · 2003 [cited by applicant]
US 20030163287A1 · Vock et al. · 2003 [cited by applicant]
US 20060112898A1 · Fjelstad et al. · 2006 [cited by applicant]
US 20100198023A1 · Yanai et al. · 2010 [cited by applicant]
US 20150096277A1 · Muffie · 2015 [cited by applicant]
US 20150302241A1 · Eineren et al. · 2015 [cited by applicant]
US 20160378981A1 · Cutler et al. · 2016 [cited by applicant]
US 20170150701A1 · Gilmore et al. · 2017 [cited by applicant]
US 20180132459A1 · Baba · 2018 [cited by examiner]
US 20190228218A1 · Barnaby et al. · 2019 [cited by applicant]
US 20190300135A1 · Troy et al. · 2019 [cited by applicant]
US 20190335720A1 · Kim · 2019 [cited by applicant]
US 20190340440A1 · Atwater et al. · 2019 [cited by applicant]
US 20200113158A1 · Rishi · 2020 [cited by applicant]
US 20200155882A1 · Tohidi et al. · 2020 [cited by applicant]
US 20200288678A1 · Howe et al. · 2020 [cited by applicant]
CA 2020618 · 1991 [cited by applicant]
CA 2455384 · 2013 [cited by applicant]
CH 709805 · 2015 [cited by applicant]
CL 201501722 · 2016 [cited by applicant]
CL 2019000039A1 · 2019 [cited by applicant]
CN 101403910 · 2009 [cited by applicant]
CN 102037909 · 2011 [cited by applicant]
CN 102630599 · 2012 [cited by applicant]
CN 103766259 · 2014 [cited by applicant]
CN 204518877 · 2015 [cited by applicant]
CN 106135070 · 2016 [cited by applicant]
CN 106538448 · 2017 [cited by applicant]
CN 107549036 · 2018 [cited by applicant]
CN 107633221 · 2018 [cited by applicant]
CN 107950427 · 2018 [cited by applicant]
CN 108029608 · 2018 [cited by applicant]
CN 108040948 · 2018 [cited by applicant]
CN 108450357 · 2018 [cited by applicant]
CN 208624391 · 2019 [cited by applicant]
DE 102016118555 · 2018 [cited by applicant]
EP 1186231 · 2002 [cited by applicant]
EP 2244934 · 2010 [cited by applicant]
EP 3484283 · 2019 [cited by applicant]
FR 1576220 · 1969 [cited by applicant]
GB 8702439 · 1987 [cited by applicant]
JP 3004788 · 1994 [cited by applicant]
JP 2002171853 · 2002 [cited by applicant]
JP 2004000086 · 2004 [cited by applicant]
KR 19990039914 · 1999 [cited by applicant]
KR 20120087212 · 2012 [cited by applicant]
NO 300401 · 1997 [cited by applicant]
NO 300401B1 · 1997 [cited by examiner]
NO 3000401 · 1997 [cited by applicant]
NO 332103 · 2012 [cited by applicant]
NO 20160199 · 2017 [cited by applicant]
NO 20160880 · 2017 [cited by applicant]
RU 2015145517 · 2017 [cited by applicant]
WO WO1990007874 · 1990 [cited by applicant]
WO WO1997019587 · 1997 [cited by applicant]
WO WO2009008733 · 2009 [cited by applicant]
WO WO2009008733A1 · 2009 [cited by examiner]
WO WO2009097057 · 2009 [cited by applicant]
WO WO2012081990 · 2012 [cited by applicant]
WO WO2014098614 · 2014 [cited by applicant]
WO WO2014179482 · 2014 [cited by applicant]
WO WO2016023071 · 2016 [cited by applicant]
WO WO2016063033 · 2016 [cited by applicant]
WO WO2017137896 · 2017 [cited by applicant]
WO WO2017153417 · 2017 [cited by applicant]
WO WO2018011744 · 2018 [cited by applicant]
WO WO2018011745 · 2018 [cited by applicant]
WO WO2018117850 · 2018 [cited by applicant]
WO WO2019002881 · 2019 [cited by applicant]
WO WO2019121851 · 2019 [cited by applicant]
WO WO2019168406 · 2019 [cited by applicant]
WO WO2019188506 · 2019 [cited by applicant]
WO WO2019212807 · 2019 [cited by applicant]
WO WO2019232247 · 2019 [cited by applicant]
WO WO2020046524 · 2020 [cited by applicant]
WO WO2020132031 · 2020 [cited by applicant]
WO WO2021006744 · 2021 [cited by applicant]
WO WO2021030237 · 2021 [cited by applicant]
WO WO2022010815 · 2022 [cited by applicant]
WO WO2020072438 · 2022 [cited by applicant]
Office Action in Chinese Appln. No. 201980065862.3, dated Jun. 5, 2023, 28 pages (with English translation). [cited by applicant]
Decision to Grant Patent in Japanese Appln. No. 2021-514534, dated Nov. 4, 2022, 5 pages (with English translation). [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/018651, dated Jun. 22, 2022, 14 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/021683, dated Jun. 27, 2022, 14 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/022250, dated Jul. 6, 2022, 15 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/022492, dated Jun. 28, 2022, 13 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/022589, dated Jul. 7, 2022, 12 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/022837, dated Aug. 2, 2022, 14 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln. No. PCT/US2020/059829, dated Feb. 25, 2021, 18 pages. [cited by applicant]
International Search Report and Written Opinion International Appln No. PCT/US2022/023831, dated Jul. 8, 2022, 13 pages. [cited by applicant]
Office Action in Canadian Appln. No. 3,113,831, dated Jun. 13, 2022, 3 pages. [cited by applicant]
Office Action in Chilean Patent Appn. 202100828, dated Aug. 6, 2022, 19 pages (with English translation). [cited by applicant]
Office Action in Chilean Patent Appn. 202100828, dated Jun. 13, 2022, 24 pages (with English translation). [cited by applicant]
Office Action in Chinese Appln. No. 201980065862.3 dated Oct. 21, 2022, 24 pages (with English translation). [cited by applicant]
Akvagroup.com [online], “Akvasmart cameras and winches,” 2016, [retrieved on Apr. 16, 2019], retrieved from: URL<https://www.akvagroup.com/Products/User%20Manuals/Camera/UK%20User%20manual%20Akvasmart%20Camera%20with%20… [cited by applicant]
Extended Search Report in European Appln. No. 22151132.2, dated May 2, 2022, 10 pages. [cited by applicant]
Maloy et al., “A spatio-temporal recurrent network for salmon feeding action recognition from underwater videos in aquaculture,” Computers and Electronics in Agriculture, Nov. 12, 2019, 9 pages. [cited by applicant]
Meidell et al., “FishNet: A Unified Embedding for Salmon Recognition,” Thesis for Master's degree in Artificial Intelligence, Norwegian University of Science and Technology, Jun. 2019, 86 pages. [cited by applicant]
Moskvyak et al., “Robust Re-identification of Manta Rays from Natural Markings by Learning Pose Invariant Embeddings,” CoRR, Feb. 2019, arXiv:1902.10847v1, 12 pages. [cited by applicant]
Odey, “AquaMesh—Design and Implementation of Smart Wireless Mesh Sensor Networks for Aquaculture,” American Journal of Networks and Communications, Jul. 2013, 8 pages. [cited by applicant]
Office Action in Canadian Appln. No. 3,087,370, dated Aug. 4, 2021, 3 pages. [cited by applicant]
Office Action in Chinese Appln. No. 201980065862.3 dated Mar. 1, 2022, 33 pages (with English translation). [cited by applicant]
Office Action in Japanese Appln. No. 2021-514534, dated May 10, 2022, 8 pages (with English translation). [cited by applicant]
PCT International Preliminary Report on Patentability in International Appln No. PCT/US2019/053986, dated Apr. 15, 2021, 10 pages. [cited by applicant]
PCT International Preliminary Report on Patentability in International Appln No. PCT/US2020/059829, dated May 27, 2022, 10 pages. [cited by applicant]
PCT International Search Report and Written Opinion in International Appln No. PCT/US2019/053986, dated Jan. 2, 2020, 30 pages. [cited by applicant]
Petrov et al., “Overview of the application of computer vision technology in fish farming,” E3S Web of Conferences, 2020, 175:02015. [cited by applicant]
Qiu et al., “Improving Transfer Learning and Squeeze-and-Excitation Networks for Small-Scale Fine-Grained Fish Image Classification,” IEEE Access, Dec. 2018, 6(31):78503-78512. [cited by applicant]
Saberloon et al., “Application of Machine Vision Systems in Aquaculture with Emphasis on Fish: State-of-the-Art and Key Issues,” Reviews in Aquaculture, Dec. 2017, 9:369-387. [cited by applicant]
Stein et al., “Consistent melanophore spot patterns allow long-term individual recognition of Atlantic salmon [cited by applicant]
towardsdatascience.com [online], “Analyzing Applications of Deep Learning in Aquaculture,” Jan. 2021, retrieved on Aug. 11, 2021, retrieved from URL<https://towardsdatascience.com/analyzing-applications-of-deep-learning… [cited by applicant]
Wang, “Robust tracking of fish schools using CNN for head identification,” Multimedia Tools and Applications, Nov. 2017, 20 pages. [cited by applicant]
Office Action in Canadian Appln. No. 3,113,831, dated Sep. 14, 2023, 6 pages. [cited by applicant]
Office Action in Canadian Appln. No. 3,113,831, dated Jan. 20, 2023, 6 pages. [cited by applicant]
Office Action in Chilean Patent Appn. 202201971, dated Sep. 5, 2023, 31 pages (with English translation). [cited by applicant]
Office Action in Japanese Appln. No. 2022-198191, dated Jan. 24, 2024, 8 pages (with English translation). [cited by applicant]
Office Action in Chilean Patent Appn. 202201971, dated Nov. 27, 2023, 7 pages (with English translation). [cited by applicant]
Office Action in Chile Appln. No. 202201971, dated May 8, 2024, 23 pages (with English translation). [cited by applicant]
Cited By (1)
US 12,466,525