IP Library Granted Patent US 12,078,533
Granted Patent B2
US 12,078,533 · App. 17/337,004 · Granted Sep 3, 2024

Underwater camera as light sensor

Inventor: Barnaby John James (Campbell, CA)
Assignee: TidalX AI Inc.
G01J1/4204G01B11/22G01J1/18G01J1/44
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,078,533
App. No.
17/337,004
Granted
Sep 3, 2024
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium that automatically performs actions in an aquaculture environment based on light sensed by underwater cameras. One of the methods includes obtaining images of a surface of water captured by a camera that faces upwards from a depth towards the surface of the water within an enclosure that encloses aquatic livestock. An ambient light metric is determined at the depth from the images of the surface of the water. A determination is made as to whether the camera satisfies one or more depth criteria. Based on determining that the depth of the camera satisfies the one or more depth criteria, it is determined that, based on the ambient light metric at the depth, one or more action criteria are satisfied, then initiating performance of an action to be performed for the aquatic livestock.

Claims (77)

1. A computer-implemented method comprising:

obtaining, by a camera system that includes an upward-facing camera and one or more side-facing cameras, images of a surface of water captured by the upward-facing camera facing towards the surface of the water from a depth within an enclosure that encloses aquatic livestock;

determining that the depth of the upward-facing camera satisfies one or more depth criteria;

in response to determining that the depth of the upward-facing camera satisfies the one or more depth criteria, determining from the images of the surface of the water that were captured by the upward-facing camera, an ambient light metric at the depth;

determining that one or more action criteria are satisfied based on the ambient light metric at the depth; and

based on determining that the one or more action criteria are satisfied based on the ambient light metric at the depth, initiating performance of an action to be performed for the aquatic livestock.

2. The computer-implemented method of claim 1 , wherein determining that the one or more action criteria are satisfied based on the ambient light metric at the depth comprises:

determining an amount of ambient light based on the ambient light metric; and

determining that the amount of ambient light at the depth exceeds a light threshold.

3. The computer-implemented method of claim 1 , wherein determining that one or more action criteria are satisfied comprises:

determining an amount of ambient light based on the ambient light metric;

generating a data set that represents the amount of ambient light determined at the depth and previous amounts of ambient light determined at the depth;

providing the data set to a machine-learning trained model that is trained to output actions for input data sets that represent amounts of ambient light determined at depths; and

receiving an indication of the action from the machine-learning trained model.

4. The computer-implemented method of claim 1 , wherein the ambient light metric is determined using an intensity value of one or more pixels of the image.

5. The computer-implemented method of claim 1 , wherein determining that the depth of the camera satisfies one or more depth criteria comprises:

comparing the depth to a depth threshold;

determining that the depth does not exceed the depth threshold; and

based on determining that the depth does not exceed the depth threshold, determining that the depth of the camera satisfies the one or more depth criteria.

6. The computer-implemented method of claim 1 , wherein determining from the images of the surface of the water, an ambient light metric at the depth comprises:

determining a first amount of ambient light at the depth from a first image of the images;

determining a second amount of ambient light at the depth from a second image of the images; and

determining the ambient light metric at the depth based on an aggregate of the first amount of ambient light at the depth from the first image of the images and the second amount of ambient light at the depth from the second image of the images.

7. The computer-implemented method of claim 1 , wherein initiating performance of an action to be performed for the aquatic livestock comprises:

transmitting an instruction to a feeder subsystem, wherein the instruction indicates to provide feed to the aquatic livestock.

8. The computer-implemented method of claim 1 , wherein initiating performance of an action to be performed for the aquatic livestock comprises:

transmitting an instruction to a feeder subsystem, wherein the instruction indicates to stop providing feed to the aquatic livestock.

9. The computer-implemented method of claim 1 , wherein determining that one or more action criteria are satisfied based on the ambient light metric at the depth comprises:

determining a predicted amount of future ambient light at the depth based on the current ambient light metric and environmental data;

determining that the predicted amount of ambient light at the depth exceeds a threshold value; and

based on determining that the predicted amount of ambient light at the depth exceeds the threshold value, determining that the one or more action criteria are satisfied.

10. The computer-implemented method of claim 1 , wherein determining that one or more action criteria are satisfied based on the amount of ambient light at the depth comprises:

determining a feeding metric based on the ambient light metric and a measurement of feed delivered over a period;

determining that the feeding metric satisfies one or more feeding metric criteria; and

based on determining that the feeding metric satisfies the one or more feeding metric criteria, determining that the one or more action criteria are satisfied.

11. A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing instructions that, when executed, cause the one or more processors to perform operations comprising:

obtaining, by a camera system that includes an upward-facing camera and one or more side-facing cameras, images of a surface of water captured by the upward-facing camera facing towards the surface of the water from a depth within an enclosure that encloses aquatic livestock;

determining that the depth of the upward-facing camera satisfies one or more depth criteria;

in response to determining that the depth of the upward-facing camera satisfies the one or more depth criteria, determining from the images of the surface of the water that were captured by the upward-facing camera, an ambient light metric at the depth;

determining that one or more action criteria are satisfied based on the ambient light metric at the depth; and

based on determining that the one or more action criteria are satisfied based on the ambient light metric at the depth, initiating performance of an action to be performed for the aquatic livestock.

12. The system of claim 11 , wherein determining that one or more action criteria are satisfied comprises:

determining an amount of ambient light based on the ambient light metric; and

determining that the amount of ambient light at the depth exceeds a light threshold.

13. The system of claim 11 , wherein determining that one or more action criteria are satisfied comprises:

determining an amount of ambient light based on the ambient light metric;

generating a data set that represents the amount of ambient light determined at the depth and previous amounts of ambient light determined at the depth;

providing the data set to a machine-learning trained model that is trained to output actions for input data sets that represent amounts of ambient light determined at depths; and

receiving an indication of the action from the machine-learning trained model.

14. The system of claim 11 , wherein determining that the depth of the camera satisfies one or more depth criteria comprises:

comparing the depth to a depth threshold;

determining that the depth does not exceed the depth threshold; and

based on determining that the depth does not exceed the depth threshold, determining that the depth of the camera satisfies the one or more depth criteria.

15. The system of claim 11 , wherein determining from the images of the surface of the water, an ambient light metric at the depth comprises:

determining a first amount of ambient light at the depth from a first image of the images;

determining a second amount of ambient light at the depth from a second image of the images; and

determining the ambient light metric at the depth based on an aggregate of the first amount of ambient light at the depth from the first image of the images and the second amount of ambient light at the depth from the second image of the images.

16. The system of claim 11 , wherein initiating performance of an action to be performed for the aquatic livestock comprises:

transmitting an instruction to a feeder subsystem, wherein the instruction indicates to provide feed to the aquatic livestock.

17. The system of claim 11 , wherein initiating performance of an action to be performed for the aquatic livestock comprises:

transmitting an instruction to a feeder subsystem, wherein the instruction indicates to stop providing feed to the aquatic livestock.

18. The system of claim 11 wherein determining that one or more action criteria are satisfied based on the ambient light metric at the depth comprises:

determining a predicted amount of future ambient light at the depth based on the current ambient light metric and environmental data;

determining that the predicted amount of ambient light at the depth exceeds a threshold value; and

based on determining that the predicted amount of ambient light at the depth exceeds the threshold value, determining that the one or more action criteria are satisfied.

19. The system of claim 11 , wherein determining that one or more action criteria are satisfied based on the amount of ambient light at the depth comprises:

determining a feeding metric based on the ambient light metric and a measurement of feed delivered over a period;

determining that the feeding metric satisfies one or more feeding metric criteria; and

based on determining that the feeding metric satisfies the one or more feeding metric criteria, determining that the one or more action criteria are satisfied.

20. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising:

obtaining, by a camera system that includes an upward-facing camera and one or more side-facing cameras, images of a surface of water captured by the upward-facing camera facing towards the surface of the water from a depth within an enclosure that encloses aquatic livestock;

determining that the depth of the upward-facing camera satisfies one or more depth criteria;

in response to determining that the depth of the upward-facing camera satisfies the one or more depth criteria, determining from the images of the surface of the water that were captured by the upward-facing camera, an ambient light metric at the depth;

determining that one or more action criteria are satisfied based on the ambient light metric at the depth; and

based on determining that the one or more action criteria are satisfied based on the ambient light metric at the depth, initiating performance of an action to be performed for the aquatic livestock.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2024
From: X DEVELOPMENT LLC
To: TIDALX AI INC.
Reel/Frame 068477/0306 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 12, 2021
From: JAMES, BARNABY JOHN
To: X DEVELOPMENT LLC
Reel/Frame 056829/0450 →
Continuity (1)
Related Publication 20220390275A1 · Dec 8, 2022