IP Library Granted Patent US 12,342,081
Granted Patent B2
US 12,342,081 · App. 18/529,983 · Granted Jun 24, 2025

Camera winch control for dynamic monitoring

Inventors: Barnaby John James (Campbell, CA); Tatiana Kichkaylo (Woodinville, WA); Peter Kimball (Mountain View, CA); Christopher Thornton (Vancouver, CA)
Assignee: TidalX AI Inc.
H04N23/695A01K29/00A01K61/13G06T7/50G06T2207/30232G06T2207/30242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,342,081
App. No.
18/529,983
Granted
Jun 24, 2025
Kind
B2
Abstract

A method for controlling a sensor subsystem, the method including receiving one or more metrics representing one or more characteristics of livestock, including one or more livestock objects, contained in an enclosure and monitored by one or more sensors coupled to a winch subsystem. The method further includes determining a position to move the one or more sensors based on the metrics and determining an instruction that includes information related to a movement of the one or more sensors. The method further includes sending the instruction to the winch subsystem to change the position of the one or more sensors.

Claims (34)

1. A computer-implemented method comprising:

receiving images of an underwater environment that were generated by an underwater camera subsystem;

providing, by the underwater camera system, features derived from the images to one or more machine-learning models that generate sets of one or more metrics regarding the underwater environment, including one or more metrics that indicate a duration that an underwater object was tracked by the underwater camera subsystem;

receiving a particular set of the one or more metrics from the one or more machine-learning models;

generating one or more winch commands based on the received, particular set of the metrics; and

transmitting the one or more winch commands to a winch that supports the underwater camera subsystem in response to receiving the images of the underwater environment.

2. The method of claim 1 , wherein the particular set of the one or more metrics includes one or more metrics that reflect a quality of a current position of the underwater camera subsystem with respect to monitoring underwater objects.

3. The method of claim 1 , wherein the particular set of the one or more metrics includes one or more metrics that indicate durations that an underwater object was independently tracked by each of multiple cameras of the underwater camera subsystem.

4. The method of claim 1 , wherein the particular set of the one or more metrics includes one or more metrics that indicate a quantity of underwater objects that were visible to one of multiple cameras of the underwater camera subsystem but that were not visible to other of the multiple cameras of the underwater camera subsystem.

5. The method of claim 1 , wherein generating the one or more winch commands comprises generating one or more particular winch commands for the winch that indicate a future position of the underwater camera subsystem, specified as a set of coordinates.

6. The method of claim 1 , wherein generating the one or more winch commands comprises generating one or more particular which commands for the winch that one or more distances to move the underwater camera system to a future position.

7. One or more non-transitory computer-readable storage media that store instructions which, when executed by one or more computer processors, cause the one or more computer processors to perform operations comprising:

receiving images of an underwater environment that were generated by an underwater camera subsystem;

providing, by the underwater camera system, features derived from the images to one or more machine-learning models that generate sets of one or more metrics regarding the underwater environment, including one or more metrics that indicate a duration that an underwater object was tracked by the underwater camera subsystem;

receiving a particular set of the one or more metrics from the one or more machine-learning models;

generating one or more winch commands based on the received, particular set of the metrics; and

transmitting the one or more winch commands to a winch that supports the underwater camera subsystem in response to receiving the images of the underwater environment.

8. The media of claim 7 , wherein the particular set of the one or more metrics includes one or more metrics that reflect a quality of a current position of the underwater camera subsystem with respect to monitoring underwater objects.

9. The media of claim 7 , wherein the particular set of the one or more metrics includes one or more metrics that indicate durations that an underwater object was independently tracked by each of multiple cameras of the underwater camera subsystem.

10. The media of claim 7 , wherein the particular set of the one or more metrics includes one or more metrics that indicate a quantity of underwater objects that were visible to one of multiple cameras of the underwater camera subsystem but that were not visible to other of the multiple cameras of the underwater camera subsystem.

11. The media of claim 7 , wherein generating the one or more winch commands comprises generating one or more particular winch commands for the winch that indicate a future position of the underwater camera subsystem, specified as a set of coordinates.

12. The media of claim 7 , wherein generating the one or more winch commands comprises generating one or more particular which commands for the winch that one or more distances to move the underwater camera system to a future position.

13. A system comprising:

one or more computer processors; and

one or more non-transitory computer-readable storage media that store instructions which, when executed by the one or more computer processors, cause the one or more computer processors to perform operations comprising:

receiving images of an underwater environment that were generated by an underwater camera subsystem;

providing, by the underwater camera system, features derived from the images to one or more machine-learning models that generate sets of one or more metrics regarding the underwater environment, including one or more metrics that indicate a duration that an underwater object was tracked by the underwater camera subsystem;

receiving a particular set of the one or more metrics from the one or more machine-learning models;

generating one or more winch commands based on the received, particular set of the metrics; and

transmitting the one or more winch commands to a winch that supports the underwater camera subsystem in response to receiving the images of the underwater environment.

14. The system of claim 13 , wherein the particular set of the one or more metrics includes one or more metrics that reflect a quality of a current position of the underwater camera subsystem with respect to monitoring underwater objects.

15. The system of claim 13 , wherein the particular set of the one or more metrics includes one or more metrics that indicate durations that an underwater object was independently tracked by each of multiple cameras of the underwater camera subsystem.

16. The system of claim 13 , wherein the particular set of the one or more metrics includes one or more metrics that indicate a quantity of underwater objects that were visible to one of multiple cameras of the underwater camera subsystem but that were not visible to other of the multiple cameras of the underwater camera subsystem.

17. The system of claim 13 , wherein generating the one or more winch commands comprises generating one or more particular winch commands for the winch that indicate a future position of the underwater camera subsystem, specified as a set of coordinates.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2024
From: X DEVELOPMENT LLC
To: TIDALX AI INC.
Reel/Frame 068477/0306 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 7, 2023
From: JAMES, BARNABY JOHN; KICHKAYLO, TATIANA; KIMBALL, PETER; THORNTON, CHRISTOPHER
To: X DEVELOPMENT LLC
Reel/Frame 065795/0200 →
Continuity (3)
Continuation 17319614 · May 13, 2021
Continuation 16785252 · Feb 7, 2020
Related Publication 20240348926A1 · Oct 17, 2024
References Cited (75)
US 5823140A · Pittet et al. · 1998 [cited by applicant]
US 10856520B1 · Kozachenok et al. · 2020 [cited by applicant]
US 20070045601A1 · Rhee · 2007 [cited by applicant]
US 20100198023A1 · Yanai et al. · 2010 [cited by applicant]
US 20140267778A1 · Webb et al. · 2014 [cited by applicant]
US 20150302241A1 · Eineren et al. · 2015 [cited by applicant]
US 20170150701A1 · Gilmore et al. · 2017 [cited by applicant]
US 20180132459A1 · Baba · 2018 [cited by applicant]
US 20180263223A1 · Kodaira et al. · 2018 [cited by applicant]
US 20190228218A1 · Barnaby et al. · 2019 [cited by applicant]
US 20190340440A1 · Atwater et al. · 2019 [cited by applicant]
US 20200107524A1 · Messana et al. · 2020 [cited by applicant]
US 20200113158A1 · Rishi · 2020 [cited by applicant]
US 20200155882A1 · Tohidi et al. · 2020 [cited by applicant]
US 20200288678A1 · Howe et al. · 2020 [cited by applicant]
US 20230045358A1 · Kobayashi · 2023 [cited by examiner]
CL 2019000039A1 · 2019 [cited by applicant]
CN 108040948 · 2018 [cited by applicant]
EP 2244934 · 2010 [cited by applicant]
EP 3484283 · 2019 [cited by applicant]
JP 2002171853 · 2002 [cited by applicant]
NO 300401 · 1997 [cited by applicant]
NO 20160199 · 2017 [cited by applicant]
NO 345829 · 2019 [cited by applicant]
WO WO1990007874 · 1990 [cited by applicant]
WO WO1997019587 · 1997 [cited by applicant]
WO WO2009008733 · 2009 [cited by applicant]
WO WO2009097057 · 2009 [cited by applicant]
WO WO2012081990 · 2012 [cited by applicant]
WO WO2014098614 · 2014 [cited by applicant]
WO WO2014179482 · 2014 [cited by applicant]
WO WO2016023071 · 2016 [cited by applicant]
WO WO2017137896 · 2017 [cited by applicant]
WO WO2018011744 · 2018 [cited by applicant]
WO WO2018011745 · 2018 [cited by applicant]
WO WO2019002881 · 2019 [cited by applicant]
WO WO2019017132 · 2019 [cited by applicant]
WO WO2019121851 · 2019 [cited by applicant]
WO WO2019188506 · 2019 [cited by applicant]
WO WO2019212807 · 2019 [cited by applicant]
WO WO2019232247 · 2019 [cited by applicant]
WO WO2019232247A1 · 2019 [cited by examiner]
WO WO2020046524 · 2020 [cited by applicant]
WO WO2020072438 · 2020 [cited by applicant]
WO WO2020132031 · 2020 [cited by applicant]
WO WO2021006744 · 2021 [cited by applicant]
WO WO2021030237 · 2021 [cited by applicant]
WO WO2022010815 · 2022 [cited by applicant]
Decision to Grant a Patent in Japanese Appln. No. 2022-538473, dated Aug. 18, 2023, 5 pages (with English translation). [cited by applicant]
Extended Search Report in European Appln. No. 22151132.2, dated May 2, 2022, 10 pages. [cited by applicant]
International Preliminary Report on Patentability in International Appln. No. PCT/US2020/059829, dated May 27, 2022, 11 pages. [cited by applicant]
International Preliminary Report on Patentability in International Appln. No. PCT/US2021/016387, dated Jul. 28, 2022, 8 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/018651, dated Jun. 22, 2022, 14 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/021683, dated Jun. 27, 2022, 14 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/022250, dated Jul. 6, 2022, 15 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/022492, dated Jun. 28, 2022, 13 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/022589, dated Jul. 7, 2022, 12 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/022837, dated Aug. 2, 2022, 14 pages. [cited by applicant]
International Search Report and Written Opinion International Appln No. PCT/US2022/023831, dated Jul. 8, 2022, 13 pages. [cited by applicant]
Maloy et al., “A spatio-temporal recurrent network for salmon feeding action recognition from underwater videos in aquaculture,” Computers and Electronics in Agriculture, Nov. 12, 2019, 9 pages. [cited by applicant]
Meidell et al., “FishNet: A Unified Embedding for Salmon Recognition,” Thesis for Master's degree in Artificial Intelligence, Norwegian University of Science and Technology, Jun. 2019, 86 pages. [cited by applicant]
Moskvyak et al., “Robust Re-identification of Manta Rays from Natural Markings by Learning Pose Invariant Embeddings,” CoRR, Feb. 2019, arXiv:1902.10847v1, 12 pages. [cited by applicant]
Odey, “AquaMesh—Design and Implementation of Smart Wireless Mesh Sensor Networks for Aquaculture,” American Journal of Networks and Communications, Jul. 2013, 8 pages. [cited by applicant]
Office Action in Canadian Appln. No. 3,087,370, dated Aug. 4, 2021, 3 pages. [cited by applicant]
Office Action in Canadian Appln. No. 3,166,208 dated Sep. 11, 2023, 3 pages. [cited by applicant]
Office Action in Chilean Appln. No. 202201772, dated Oct. 8, 2024, 26 pages (with English translation). [cited by applicant]
Office Action in Japanese Appln. No. 2022-538473, dated May 8, 2023, 6 pages (with English translation). [cited by applicant]
PCT International Search Report and Written Opinion in International Appln. No. PCT/US2020/059829, dated Feb. 25, 2021, 18 pages. [cited by applicant]
PCT International Search Report and Written Opinion in International Appln. No. PCT/US2021/016387, dated May 3, 2021, 14 pages. [cited by applicant]
Petrov et al., “Overview of the application of computer vision technology in fish farming,” E3S Web of Conferences, 2020, 175:02015. [cited by applicant]
Qiu et al., “Improving Transfer Learning and Squeeze-and-Excitation Networks for Small-Scale Fine-Grained Fish Image Classification,” IEEE Access, Dec. 2018, 6(31):78503-78512. [cited by applicant]
Saberloon et al., “Application of Machine Vision Systems in Aquaculture with Emphasis on Fish: State-of-the-Art and Key Issues,” Reviews in Aquaculture, Dec. 2017, 9:369-387. [cited by applicant]
Stein et al., “Consistent melanophore spot patterns allow long-term individual recognition of Atlantic salmon Salmo Salar,” Journal of Fish Biology, Nov. 2017, 91(6):1699-1712. [cited by applicant]
towardsdatascience.com [online], “Analyzing Applications of Deep Learning in Aquaculture,” Jan. 2021, retrieved on Aug. 11, 2021, retrieved from URL<https://towardsdatascience.com/analyzing-applications-of-deep-learning… [cited by applicant]
Wang, “Robust tracking of fish schools using CNN for head identification,” Multimedia Tools and Applications, Nov. 2017, 20 pages. [cited by applicant]