IP Library Granted Patent US 12,487,504
Granted Patent B2
US 12,487,504 · App. 18/139,173 · Granted Dec 2, 2025

Underwater camera system controller for aquaculture behavior observation

Inventors: Matthew Stokes (Centennial, CO); Thomas Robert Swanson (Sunnyvale, CA); Harrison Pham (Sunnyvale, CA); Riva Gulassa (Sunnyvale, CA)
Assignee: TidalX AI Inc.
G03B17/08G01S5/0027G03B17/561H04N23/66H04N23/695A01K61/80G03B2217/002G03B2217/18
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,487,504
App. No.
18/139,173
Granted
Dec 2, 2025
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for automated underwater camera system control for aquaculture systems. An underwater camera system includes (i) a line on which the underwater camera system is mounted, the line detachably affixed to a feeder that provides feed for aquatic livestock, (ii) a sensor manager, (iii) one or more sensors that are managed by the sensor manager, (iv) a line navigation controller, and (v) a first actuator for controlling a distance between the feeder and the underwater camera system. The one or more sensors obtain sensor data and the line navigation controller of the underwater camera system determines a distance to position the underwater camera system beneath the feeder to obtain additional sensor data. The line navigation controller transmits a first message to the first actuator to position the underwater camera system at the determined distance beneath the feeder.

Claims (67)

1 . A computer-implemented method comprising:

determining, by a sensor manager of an underwater camera system that includes (i) a line on which the underwater camera system is mounted, the line detachably affixed to a feeder that provides feed for aquatic livestock, (ii) the sensor manager, (iii) one or more sensors that are managed by the sensor manager, (iv) a line navigation controller, and (v) a first actuator for controlling a distance between the feeder and the underwater camera system, to obtain sensor data using the one or more sensors;

obtaining, by the one or more sensors of the underwater camera system, the sensor data;

determining, by the line navigation controller, a distance of the underwater camera system from the feeder at which the underwater camera system is to be positioned beneath the feeder to obtain additional sensor data; and

transmitting, by the line navigation controller, a first message configured to instruct the first actuator to position the underwater camera system at the determined distance beneath the feeder.

2 . The computer-implemented method of claim 1 , wherein the distance to position the underwater camera system beneath the feeder is determined based at least in part on the sensor data.

3 . The computer-implemented method of claim 2 , further comprising determining, based on the sensor data, a current distance between the underwater camera system and a feed pellet column dispersed by the feeder; and

wherein the distance to position the underwater camera system beneath the feeder is determined based at least in part on the current distance between the underwater camera system and the feed pellet column.

4 . The computer-implemented method of claim 3 , wherein the distance to position the underwater camera system beneath the feeder is determined based at least in part on a current vertical position of the underwater camera system and the sensor data.

5 . The computer-implemented method of claim 4 :

wherein the underwater camera system further includes: 1) a second actuator for controlling an amount and type of drag imparted by a drag mechanism of the underwater camera system, and 2) a drag mechanism controller; and

wherein the method further comprises:

determining, by the drag mechanism controller of the underwater camera system, the amount and type of drag; and

transmitting, by the drag mechanism controller, a second message configured to instruct the second actuator to adjust at least one drag mechanism to create the amount and type of drag.

6 . The computer-implemented method of claim 5 , wherein the amount and type of drag is determined based on one or more of the current vertical position of the underwater camera system, the current distance between the underwater camera system and the feed pellet column, and the sensor data.

7 . The computer-implemented method of claim 1 :

wherein the sensors of the underwater camera system are included in a rotation mechanism that is rotatable around the underwater camera system; and

wherein the method further comprises:

determining, by the sensor manager, an amount and type of rotation; and

transmitting, by the sensor manager, a message to the rotation mechanism to rotate the rotation mechanism by the amount and type of rotation.

8 . An underwater camera system comprising:

a line on which the underwater camera system is mounted, wherein the line is detachably affixed to a feeder that provides feed for aquatic livestock;

a sensor manager,

one or more sensors that are managed by the sensor manager,

a line navigation controller,

a first actuator for controlling a distance between the feeder and the underwater camera system; and

one or more processors that are configured to:

determine to obtain sensor data using one or more of the sensors;

obtain the sensor data;

determine a distance of the underwater camera system from the feeder at which the underwater camera system is to be positioned beneath the feeder to obtain additional sensor data; and

transmit a first message configured to instruct the first actuator to position the underwater camera system at the determined distance beneath the feeder.

9 . The underwater camera system of claim 8 , wherein the distance to position the underwater camera system beneath the feeder is determined based at least in part on the sensor data.

10 . The underwater camera system of claim 9 , wherein the one or more processors are further configured to determine, based on the sensor data, a current distance between the underwater camera system and a feed pellet column dispersed by the feeder; and

wherein the distance to position the underwater camera system beneath the feeder is determined based at least in part on the current distance between the underwater camera system and the feed pellet column.

11 . The underwater camera system of claim 10 , wherein the distance to position the underwater camera system beneath the feeder is determined based at least in part on a current vertical position of the underwater camera system and the sensor data.

12 . The underwater camera system of claim 11 , further comprising:

a second actuator for controlling an amount and type of drag of the underwater camera system;

at least one drag mechanism; and

a drag mechanism controller; and

wherein the drag mechanism controller is configured to:

determine the amount and type of drag; and

transmit a second message to the second actuator to adjust at least one drag mechanism to create the amount and type of drag.

13 . The underwater camera system of claim 12 , wherein the amount and type of drag is determined based on one or more of the current vertical position of the underwater camera system, the current distance between the underwater camera system and the feed pellet column, and the sensor data.

14 . The underwater camera system of claim 8 :

wherein the sensors of the underwater camera system are included in a rotation mechanism that is rotatable around the underwater camera system; and

wherein the sensor manager is configured to:

determine an amount and type of rotation; and

transmit a message to the rotation mechanism to rotate the rotation mechanism by the amount and type of rotation.

15 . One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

determining, by a sensor manager of an underwater camera system that includes (i) a line on which the underwater camera system is mounted, the line detachably affixed to a feeder that provides feed for aquatic livestock, (ii) the sensor manager, (iii) one or more sensors that are managed by the sensor manager, (iv) a line navigation controller, and (v) a first actuator for controlling a distance between the feeder and the underwater camera system, to obtain sensor data using the one or more sensors;

obtaining, by the one or more sensors of the underwater camera system, the sensor data;

determining, by the line navigation controller, a distance of the underwater camera system from the feeder at which the underwater camera system is to be positioned beneath the feeder to obtain additional sensor data; and

transmitting, by the line navigation controller, a first message configured to instruct the first actuator to position the underwater camera system at the determined distance beneath the feeder.

16 . The media of claim 15 , wherein the operations comprise determining, based on the sensor data, a current distance between the underwater camera system and a feed pellet column dispersed by the feeder; and

wherein the distance to position the underwater camera system beneath the feeder is determined based at least in part on the current distance between the underwater camera system and the feed pellet column.

17 . The media of claim 16 , wherein the distance to position the underwater camera system beneath the feeder is determined based at least in part on a current vertical position of the underwater camera system and the sensor data.

18 . The media of claim 17 :

wherein the underwater camera system further includes: 1) a second actuator for controlling an amount and type of drag imparted by a drag mechanism of the underwater camera system, and 2) a drag mechanism controller; and

wherein the operations further comprise:

determining, by the drag mechanism controller of the underwater camera system, the amount and type of drag; and

transmitting, by the drag mechanism controller, a second message to the second actuator to adjust at least one drag mechanism to create the amount and type of drag.

19 . The media of claim 18 , wherein the amount and type of drag is determined based on one or more of the current vertical position of the underwater camera system, the current distance between the underwater camera system and the feed pellet column, and the sensor data.

20 . The media of claim 15 :

wherein the sensors of the underwater camera system are included in a rotation mechanism that is rotatable around the underwater camera system; and

wherein the method further comprises:

determining, by the sensor manager, an amount and type of rotation; and

transmitting, by the sensor manager, a message to the rotation mechanism to rotate the rotation mechanism by the amount and type of rotation.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2024
From: X DEVELOPMENT LLC
To: TIDALX AI INC.
Reel/Frame 068477/0306 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 28, 2023
From: STOKES, MATTHEW; SWANSON, THOMAS ROBERT; PHAM, HARRISON; GULASSA, RIVA
To: X DEVELOPMENT LLC
Reel/Frame 063481/0028 →
Continuity (2)
Provisional Application 63415935 · Oct 13, 2022
Related Publication 20240126145A1 · Apr 18, 2024
References Cited (49)
US 10856520B1 · Kozachenok et al. · 2020 [cited by applicant]
US 11297247B1 · James · 2022 [cited by examiner]
US 20170150701A1 · Gilmore et al. · 2017 [cited by applicant]
US 20190228218A1 · Barnaby et al. · 2019 [cited by applicant]
US 20190340440A1 · Atwater et al. · 2019 [cited by applicant]
US 20200107524A1 · Messana et al. · 2020 [cited by applicant]
US 20200155882A1 · Tohidi et al. · 2020 [cited by applicant]
US 20200288678A1 · Howe et al. · 2020 [cited by applicant]
US 20210360906A1 · Yao · 2021 [cited by examiner]
US 20220217952A1 · Messana · 2022 [cited by examiner]
CN 108040948 · 2018 [cited by applicant]
EP 2244934 · 2010 [cited by applicant]
EP 3484283 · 2019 [cited by applicant]
JP 2002171853 · 2002 [cited by applicant]
NO 300401 · 1997 [cited by applicant]
WO WO1990007874 · 1990 [cited by applicant]
WO WO1997019587 · 1997 [cited by applicant]
WO WO2009008733 · 2009 [cited by applicant]
WO WO2009097057 · 2009 [cited by applicant]
WO WO2014179482 · 2014 [cited by applicant]
WO WO2018011744 · 2018 [cited by applicant]
WO WO2019002881 · 2019 [cited by applicant]
WO WO2019121851 · 2019 [cited by applicant]
WO WO2019188506 · 2019 [cited by applicant]
WO WO2019232247 · 2019 [cited by applicant]
WO WO2020046524 · 2020 [cited by applicant]
WO WO2020132031 · 2020 [cited by applicant]
WO WO2021006744 · 2021 [cited by applicant]
WO WO2021030237 · 2021 [cited by applicant]
WO WO2022010815 · 2022 [cited by applicant]
WO WO2020072438 · 2022 [cited by applicant]
Extended Search Report in European Appln. No. 22151132.2, dated May 2, 2022, 10 pages. [cited by applicant]
International Preliminary Report on Patentability in International AppIn. No. PCT/US2020/059829, dated May 27, 2022, 11 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/018651, dated Jun. 22, 2022, 14 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/021683, dated Jun. 27, 2022, 14 pages. [cited by applicant]
International Search Report and Written Opinion in International AppIn No. PCT/US2022/022250, dated Jul. 6, 2022, 15 pages. [cited by applicant]
International Search Report and Written Opinion in International AppIn No. PCT/US2022/022492, dated Jun. 28, 2022, 13 pages. [cited by applicant]
International Search Report and Written Opinion in International AppIn No. PCT/US2022/022589, dated Jul. 7, 2022, 12 pages. [cited by applicant]
International Search Report and Written Opinion in International AppIn No. PCT/US2022/022837, dated Aug. 2, 2022, 14 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/023831, dated Jul. 8, 2022, 13 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln. No. PCT/US2020/059829, dated Feb. 25, 2021, 18 pages. [cited by applicant]
Meidell et al., “FishNet: A Unified Embedding for Salmon Recognition,” Thesis for Master's degree in Artificial Intelligence, Norwegian University of Science and Technology, Jun. 2019, 86 pages. [cited by applicant]
Moskvyak et al., “Robust Re-identification of Manta Rays from Natural Markings by Learning Pose Invariant Embeddings,” CoRR, Feb. 2019, arXiv:1902.10847v1, 12 pages. [cited by applicant]
Petrov et al., “Overview of the application of computer vision technology in fish farming,” E3S Web of Conferences, 2020, 175:02015. [cited by applicant]
Qiu et al., “Improving Transfer Learning and Squeeze-and-Excitation Networks for Small-Scale Fine-Grained Fish Image Classification,” IEEE Access, Dec. 2018, 6(31):78503-78512. [cited by applicant]
Saberloon et al., “Application of machine vision systems in aquaculture with emphasis on fish: state-of-the-art and key issues,” Reviews in Aquaculture, Dec. 2017, 9:369-387. [cited by applicant]
Stein et al., “Consistent melanophore spot patterns allow long-term individual recognition of Atlantic salmon [cited by applicant]
Towardsdatascience.com [online], “Analyzing Applications of Deep Learning in Aquaculture,” Jan. 23, 2021, retrieved on Jan. 30, 2024, retrieved from URL<https://towardsdatascience.com/analyzing-applications-of-deep-lear… [cited by applicant]
Wang, “Robust tracking of fish schools using CNN for head identification,” Multimedia Tools and Applications, Nov. 2017, 19 pages. [cited by applicant]