IP Library Granted Patent US 12,051,222
Granted Patent B2
US 12,051,222 · App. 17/374,257 · Granted Jul 30, 2024

Camera calibration for feeding behavior monitoring

Inventors: Grace Taixi Brentano (Redwood City, CA); Barnaby John James (Campbell, CA); Laura Valentine Chrobak (Menlo Park, CA); Zhaoying Yao (Palo Alto, CA)
Assignee: X Development LLC
G06T7/80A01K61/85A01K61/90A01K61/10G06T2207/30242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,051,222
App. No.
17/374,257
Granted
Jul 30, 2024
Kind
B2
Abstract

In one aspect, there is provided a method that includes receiving, by a control system having (i) a first camera configured to obtain an image of a scene, (ii) a winch controller, and (iii) a feeding system configured to deliver a feed to aquaculture, instructions to initiate a calibration of the first camera, determining a calibration state of the first camera, determining a sequence of calibration steps based on the calibration state of the first camera, and executing the sequence of calibration steps to calibrate the first camera.

Claims (71)

1. A method comprising:

receiving, by a control system that includes (i) a first camera that is initially uncalibrated and that is configured to obtain an image of a scene, (ii) a winch controller, and (iii) a feeding system configured to deliver a feed to aquaculture, instructions to initiate a calibration of the first camera;

selecting a warm calibration state for the first camera;

determining a sequence of calibration steps based on the selected state for the first camera; and

executing the sequence of calibration steps to calibrate the first camera, comprising:

determining an estimated calibration position;

determining a first range based on the estimated calibration position;

moving, by the winch controller, the first camera across the first range at a first speed;

obtaining, by the first camera, the image of the scene; and

determining, based on the image of the scene obtained by the first camera, whether the feed delivered by the feeding system is above a threshold.

2. The method of claim 1 , wherein the image of the scene obtained by the first camera includes at least one frame, and wherein the threshold specifies a count of the feed detected in the at least one frame.

3. The method of claim 1 , wherein the control system further includes a second camera configured to obtain the image of the scene, and wherein the threshold specifies an aggregate of (i) a first count of feed detected in the image of the scene obtained by the first camera and (ii) a second count of feed detected in the image of the scene obtained by the second camera.

4. The method of claim 1 , further comprising:

determining based on the image of the scene obtained by the first camera that the feed delivered by the feeding system is above the threshold; and

determining that the first camera is calibrated successfully.

5. The method of claim 1 , further comprising:

determining, from the image of the scene obtained by the first camera that the feed delivered by the feeding system is below the threshold; and

selecting a different calibration state for the first camera.

6. A method comprising:

receiving, by a control system that includes (i) a first camera configured to obtain an image of a scene, (ii) a winch controller, and (iii) a feeding system configured to deliver a feed to aquaculture, instructions to initiate a calibration of the first camera;

selecting a fast full calibration state for the first camera;

determining a sequence of calibration steps based on the selected state for the first camera; and

executing the sequence of calibration steps to calibrate the first camera, comprising:

determining a second range based on a first range associated with a warm calibration state, wherein the second range is larger than the first range;

moving, by the winch controller, the first camera across the second range at a first speed;

obtaining, by the first camera, the image of the scene; and

determining, based on the image of the scene obtained by the first camera, whether the feed delivered by the feeding system is above a threshold.

7. The method of claim 6 , further comprising:

determining based on the image of the scene obtained by the first camera that the feed delivered by the feeding system is above the threshold; and

determining that the first camera is calibrated successfully.

8. The method of claim 6 , further comprising:

determining, from the image of the scene obtained by the first camera, that the feed delivered by the feeding system is below the threshold; and

selecting a different calibration state for the first camera.

9. A method comprising:

receiving, by a control system that includes (i) a first camera configured to obtain an image of a scene, (ii) a winch controller, and (iii) a feeding system configured to deliver a feed to aquaculture, instructions to initiate a calibration of the first camera;

selecting a slow full calibration state for the first camera;

determining a sequence of calibration steps based on the selected state for the first camera; and

executing the sequence of calibration steps to calibrate the first camera, comprising:

moving, by the winch controller, the first camera across a second range at a second speed, wherein the second range is associated with a fast-full calibration state, and wherein the second speed is slower than a first speed associated with the fast-full calibration state;

obtaining, by the first camera, the image of the scene; and

determining, based on the image of the scene obtained by the first camera, whether the feed delivered by the feeding system is above a threshold.

10. The method of claim 9 , further comprising:

determining based on the image of the scene obtained by the first camera that the feed delivered by the feeding system is above the threshold; and

determining that the first camera is calibrated successfully.

11. One or more non-transitory computer-readable storage medium coupled to one or more processors that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving, by a control system that includes (i) a first camera that is initially uncalibrated and that is configured to obtain an image of a scene, (ii) a winch controller, and (iii) a feeding system configured to deliver a feed to aquaculture, instructions to initiate a calibration of the first camera;

selecting a warm calibration state for the first camera;

determining a sequence of calibration steps based on the selected state for the first camera; and

executing the sequence of calibration steps to calibrate the first camera, comprising:

determining an estimated calibration position;

determining a first range based on the estimated calibration position;

moving, by the winch controller, the first camera across the first range at a first speed;

obtaining, by the first camera, the image of the scene; and

determining, based on the image of the scene obtained by the first camera, whether the feed delivered by the feeding system is above a threshold.

12. The one or more non-transitory computer-readable storage medium of claim 11 , wherein the image of the scene obtained by the first camera includes at least one frame, and wherein the threshold specifies a count of the feed detected in the at least one frame.

13. The one or more non-transitory computer-readable storage medium of claim 11 , wherein the control system further includes a second camera configured to obtain the image of the scene, and wherein the threshold specifies an aggregate of (i) a first count of feed detected in the image of the scene obtained by the first camera and (ii) a second count of feed detected in the image of the scene obtained by the second camera.

14. A control system comprising:

a first camera that is initially uncalibrated and that is configured to obtain an image of a scene;

a winch controller;

a feeding system configured to deliver a feed to aquaculture;

one or more computers; and

one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

receiving instructions to initiate a calibration of the first camera;

selecting a warm calibration state for the first camera;

determining a sequence of calibration steps based on the selected state for the first camera; and

executing the sequence of calibration steps to calibrate the first camera, comprising:

determining an estimated calibration position;

determining a first range based on the estimated calibration position;

moving, by the winch controller, the first camera across the first range at a first speed;

obtaining, by the first camera, the image of the scene; and

determining, based on the image of the scene obtained by the first camera, whether the feed delivered by the feeding system is above a threshold.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2024
From: X DEVELOPMENT LLC
To: TIDALX AI INC.
Reel/Frame 068477/0306 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 1, 2021
From: BRENTANO, GRACE TAIXI; JAMES, BARNABY JOHN; CHROBAK, LAURA VALENTINE; YAO, ZHAOYING
To: X DEVELOPMENT LLC
Reel/Frame 057354/0600 →
Continuity (1)
Related Publication 20230017422A1 · Jan 19, 2023