IP Library Granted Patent US 12,279,599
Granted Patent B2
US 12,279,599 · App. 18/349,449 · Granted Apr 22, 2025

Turbidity determination using computer vision

Inventors: Laura Valentine Chrobak (Menlo Park, CA); Peter Kimball (Mountain View, CA); Barnaby John James (Campbell, CA); Julia Black Ling (Menlo Park, CA)
Assignee: TidalX AI Inc.
A01K61/95A23K50/80G01N21/532G01N21/534G01N33/18G06F18/22G06N20/00G06V40/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,279,599
App. No.
18/349,449
Granted
Apr 22, 2025
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, that generate from a first pair and a second pair of images of livestock that are within an enclosure and that are taken at different times using a stereoscopic camera, at least two distance distributions of the aquatic livestock within the enclosure. The distance distributions can be used to determine a measure associated with an optical property of the water within the enclosure. A signal associated with the measure can be provided.

Claims (59)

1. A computer-implemented method comprising:

receiving a first pair of images of an aquaculture environment, the first pair of images captured by a camera system at a first time;

receiving a second pair of images of the aquaculture environment, the second pair of images captured by the camera system at a second time different from the first time;

generating, from each of the first pair of images and the second pair of images, first and second respective distance distributions of one or more objects in the aquaculture environment, wherein each of the first and second respective distance distributions represents counts of objects at respective distances from the camera system;

determining, based on a comparison of the first and second distance distributions, a measure associated with an optical property of water in the aquaculture environment; and

generating, based on the measure, a signal configured to activate one or more systems associated with the aquaculture environment.

2. The method of claim 1 , wherein the optical property of the water is turbidity.

3. The method of claim 1 , wherein the signal is associated with a change in operation of a feeding subsystem.

4. The method of claim 1 , wherein determining the measure associated with the optical property of the water further comprises:

determining, using computer vision, an identification of each of the one or more objects in the environment; and

determining, based on the identification of each of the one or more objects in aquaculture environment, the measure associated with the optical property of the water.

5. The method of claim 1 , further comprising:

obtaining criteria associated with the measure associated with the optical property of the water;

determining whether the criteria are satisfied; and

in response to determining that the criteria is satisfied, providing the signal.

6. The method of claim 5 , wherein determining whether the criteria are satisfied includes evaluating the signal using at least one of (i) at least one rule, or (ii) at least one trained machine learning model.

7. A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:

receiving a first pair of images of an aquaculture environment, the first pair of images captured by a camera system at a first time;

receiving a second pair of images of the aquaculture environment, the second pair of images captured by the camera system at a second time different from the first time;

generating, from each of the first pair of images and the second pair of images, first and second respective distance distributions of one or more objects in the aquaculture environment, wherein each of the first and second respective distance distributions represents counts of objects at respective distances from the camera system;

determining, based on a comparison of the first and second distance distributions, a measure associated with an optical property of water in the aquaculture environment; and

generating, based on the measure, a signal configured to active one or more systems associated with the aquaculture environment.

8. The system of claim 7 , wherein the optical property of the water is turbidity.

9. The system of claim 7 , wherein the signal is associated with a change in operation of a feeding subsystem.

10. The system of claim 7 , wherein determining the measure associated with the optical property of the water further comprises:

determining, using computer vision, an identification of each of the one or more objects in the aquaculture environment; and

determining, based on the identification of each of the one or more object in the aquaculture environment, the measure associated with the optical property of the water.

11. The system of claim 7 , wherein providing the signal further comprises:

obtaining criteria associated with the measure associated with the optical property of the water;

determining whether the criteria are satisfied; and

in response to determining that the criteria is satisfied, providing the signal.

12. One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

receiving a first pair of images of an aquaculture environment, the first pair of images captured by a camera system at a first time;

receiving a second pair of images of the aquaculture environment, the second pair of images captured by the camera system at a second time different from the first time;

generating, from each of the first pair of images and the second pair of images, first and second respective distance distributions of one or more objects in the aquaculture environment, wherein each of the first and second respective distance distributions represents counts of objects at respective distances from the camera system;

determining, based on a comparison of the first and second distance distributions, a measure associated with an optical property of water in the aquaculture environment; and

generating, based on the measure, a signal configured to activate one or more systems associated with the aquaculture environment.

13. The one or more non-transitory computer-readable storage media of claim 12 , wherein the optical property of the water is turbidity.

14. The one or more non-transitory computer-readable storage media of claim 12 , wherein the signal is associated with a change in operation of a feeding subsystem.

15. The one or more non-transitory computer-readable storage media of claim 12 , wherein determining the measure associated with a property of the water further comprises:

determining, using computer vision, an identification of each of the one or more objects in the environment; and

determining, from the identification of each of the one or more objects in the environment, the measure associated with the optical property of the water.

16. The one or more non-transitory computer-readable storage media of claim 12 , wherein the operations further comprise:

obtaining criteria associated with the measure associated with the optical property of the water;

determining whether the criteria are satisfied; and

in response to determining that the criteria is satisfied, providing the signal.

17. The method of claim 1 , wherein generating the first and second respective distance distributions of one or more objects in the aquaculture environment comprises, for each of the first and second respective distance distributions:

identifying, by a model configured to perform object recognition, the one or more objects in the environment;

determining, by one or more computing devices, a set of relative distances for the one or more objects from the respective pair of images, wherein each relative distance in the set of relative distances is an estimated distance between an object of the one or more objects and the camera system; and

generating, by the one or more computing devices, a set of absolute distances for the one or more objects based on the set of relative distances and parameters of the camera system that captured the respective pair of images, wherein each absolute distance in the set of absolute distances represents an adjustment of the respective relative distance for an object in the one or more objects, wherein the adjustment of the respective relative distance is based on the parameters of the camera system.

18. The method of claim 17 , wherein determining the measure associated with the optical property of the water comprises:

determining a first median distance for the first set of absolute distances for the first distance distribution and a second median distance for the second set of absolute distances for the second distance distribution;

comparing the first median distance to the second median distance;

based on the comparison of the first median distance to the second median distance, determining a rate of change for the optical property of the water; and

determining the measure based on the rate of change for the optical property of the water.

19. The method of claim 1 , wherein the signal is configured to adjust at least one lighting device associated with the aquaculture environment, trigger one or more alerts in a maintenance system associated with the aquaculture environment, affect an operation of a parasite detection system associated with the aquaculture environment, affect an operation of a navigation system associated with the camera device.

20. The method of claim 1 , wherein the signal is configured to affect an operation of a biomass estimation system, and wherein affecting the operation of the biomass estimation system comprises:

assigning a confidence score of measurements captured by the biomass estimation system based on conditions of the aquaculture environment; and

adjusting, based on the signal and the confidence score, a duration of time for capturing measurements by the biomass estimation system.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2024
From: X DEVELOPMENT LLC
To: TIDALX AI INC.
Reel/Frame 068477/0306 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 12, 2023
From: CHROBAK, LAURA VALENTINE; KIMBALL, PETER; JAMES, BARNABY JOHN; LING, JULIA BLACK
To: X DEVELOPMENT LLC
Reel/Frame 064266/0913 →
Continuity (2)
Continuation 17379893 · Jul 19, 2021
Related Publication 20240147968A1 · May 9, 2024
References Cited (81)
US 5828458A · Taylor et al. · 1998 [cited by applicant]
US 7142299B2 · Tokhtuev et al. · 2006 [cited by applicant]
US 7355194B2 · Tobimatsu · 2008 [cited by applicant]
US 10856520B1 · Kozachenok et al. · 2020 [cited by applicant]
US 20060055927A1 · Feng · 2006 [cited by applicant]
US 20070151522A1 · Brauman · 2007 [cited by applicant]
US 20130044325A1 · Koyanagi · 2013 [cited by applicant]
US 20150308948A1 · Robitaille et al. · 2015 [cited by applicant]
US 20170150701A1 · Gilmore et al. · 2017 [cited by applicant]
US 20190228218A1 · Barnaby et al. · 2019 [cited by applicant]
US 20190340440A1 · Atwater et al. · 2019 [cited by applicant]
US 20200107524A1 · Messana et al. · 2020 [cited by applicant]
US 20200155882A1 · Tohidi et al. · 2020 [cited by applicant]
US 20200288678A1 · Howe et al. · 2020 [cited by applicant]
US 20210041368A1 · Lu et al. · 2021 [cited by applicant]
US 20210085165A1 · Rauniyar et al. · 2021 [cited by applicant]
US 20210329892A1 · Kozachenok et al. · 2021 [cited by applicant]
US 20220237799A1 · Price et al. · 2022 [cited by applicant]
US 20220400657A1 · Humphreys · 2022 [cited by applicant]
CL 2019000039A1 · 2019 [cited by applicant]
CN 103499539 · 2014 [cited by applicant]
CN 106018408 · 2016 [cited by applicant]
CN 108040948 · 2018 [cited by applicant]
EP 2244934 · 2010 [cited by applicant]
EP 3484283 · 2019 [cited by applicant]
GB 2568501 · 2019 [cited by applicant]
JP 2002171853 · 2002 [cited by applicant]
NO 300401 · 1997 [cited by applicant]
NO 20160199 · 2017 [cited by applicant]
WO WO1990007874 · 1990 [cited by applicant]
WO WO1997019587 · 1997 [cited by applicant]
WO WO2009008733 · 2009 [cited by applicant]
WO WO2009097057 · 2009 [cited by applicant]
WO WO2012081990 · 2012 [cited by applicant]
WO WO2014179482 · 2014 [cited by applicant]
WO WO2015161748 · 2015 [cited by applicant]
WO WO2017137896 · 2017 [cited by applicant]
WO WO2017175261 · 2017 [cited by applicant]
WO WO2018011744 · 2018 [cited by applicant]
WO WO2018011745 · 2018 [cited by applicant]
WO WO2019002881 · 2019 [cited by applicant]
WO WO2019121851 · 2019 [cited by applicant]
WO WO2019188506 · 2019 [cited by applicant]
WO WO2019232247 · 2019 [cited by applicant]
WO WO2020046524 · 2020 [cited by applicant]
WO WO2020132031 · 2020 [cited by applicant]
WO WO2021006744 · 2021 [cited by applicant]
WO WO2021030237 · 2021 [cited by applicant]
WO WO2022010815 · 2022 [cited by applicant]
WO WO2020072438 · 2022 [cited by applicant]
Extended Search Report in European Appln. No. 22151132.2, dated May 2, 2022, 10 pages. [cited by applicant]
Garcia et al., “Monitoring and control sensor system for fish feeding in marine fish farms,” IET Commun., Aug. 2011, 5(12):1682-1690. [cited by applicant]
International Preliminary Report on Patentability in International Appln. No. PCT/US2020/059829, dated May 27, 2022, 11 pages. [cited by applicant]
International Preliminary Report on Patentability in International Appln. No. PCT/US2022/032756, dated Feb. 1, 2024, 10 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/018651, dated Jun. 22, 2022, 14 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/021683, dated Jun. 27, 2022, 14 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/022250, dated Jul. 6, 2022, 15 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/022492, dated Jun. 28, 2022, 13 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/022589, dated Jul. 7, 2022, 12 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/022837, dated Aug. 2, 2022, 14 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln No. PCT/US2022/023831, dated Jul. 8, 2022, 13 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln. No. PCT/US2020/059829, dated Feb. 25, 2021, 18 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln. No. PCT/US2022/032756, dated Nov. 18, 2022, 18 pages. [cited by applicant]
Invitation to Pay Additional Fees in International Appln. No. PCT/US2022/032756, dated Sep. 28, 2022, 10 pages. [cited by applicant]
Koehrsen (“Beyond Accuracy: Precision and Recall”, willkoehrsen.github.io 2018) (Year: 2018). [cited by applicant]
Liu et al., “A Review of Turbidity Detection Based on Computer Vision,” IEEE Access, Oct. 2018, 6:60586-60604. [cited by applicant]
Maloy et al., “A spatio-temporal recurrent network for salmon feeding action recognition from underwater videos in aquaculture,” Computers and Electronics in Agriculture, Nov. 12, 2019, 9 pages. [cited by applicant]
Markussen, “Parchar—Characterization of Suspended Particles Through Image Processing in Matlab,” Journal of Open Research Software, Jul. 1, 2016, 4(1):e26. [cited by applicant]
Meidell et al., “FishNet: A Unified Embedding for Salmon Recognition,” Thesis for Master's degree in Artificial Intelligence, Norwegian University of Science and Technology, Jun. 2019, 86 pages. [cited by applicant]
Moskvyak et al., “Robust Re-identification of Manta Rays from Natural Markings by Learning Pose Invariant Embeddings,” CoRR, Feb. 2019, arXiv:1902.10847vl, 12 pages. [cited by applicant]
Naval et al., “FishDrop: Estimation of reef fish population density and biomass using stereo cameras,” 2016 Techno-Ocean (Techno-Ocean), Oct. 2016, pp. 527-531. [cited by applicant]
Odey, “AquaMesh—Design and Implementation of Smart Wireless Mesh Sensor Networks for Aquaculture,” American Journal of Networks and Communications, Jul. 2013, 8 pages. [cited by applicant]
Petrov et al., “Overview of the application of computer vision technology in fish farming.” E3S Web of Conferences, 2020, 175:02015. [cited by applicant]
Qiu et al., “Improving Transfer Learning and Squeeze-and-Excitation Networks for Small-Scale Fine-Grained Fish Image Classification,” IEEE Access, Dec. 2018, 6(31):78503-78512. [cited by applicant]
Roser er al., “Simultaneous underwater visibility assessment, enhancement and improved stereo,” 2014 IEEE International Conference on Robotics and Automation (ICRA), May 2014, pp. 3840-3847. [cited by applicant]
Saberloon et al., “Application of Machine Vision Systems in Aquaculture with Emphasis on Fish: State-of-the-Art and Key Issues,” Reviews in Aquaculture, Dec. 2017, 9:369-387. [cited by applicant]
Stein et al., “Consistent melanophore spot patterns allow long-term individual recognition of Atlantic salmon [cited by applicant]
towardsdatascience.com [online], “Analyzing Applications of Deep Learning in Aquaculture,” Jan. 2021, retrieved on Aug. 11, 2021, retrieved from URL<https://towardsdatascience.com/analyzing-applications-of-deep-learning… [cited by applicant]
Turbidity and Water, USGS (Year: 2018). [cited by applicant]
Turbidity, Wikipedia (Year: 2004). [cited by applicant]
Wang, “Robust tracking of fish schools using CNN for head identification,” Multimedia Tools and Applications, Nov. 2017, 20 pages. [cited by applicant]