IP Library Granted Patent US 11,089,227
Granted Patent B1
US 11,089,227 · App. 16/785,252 · Granted Aug 10, 2021

Camera winch control for dynamic monitoring

Inventors: Barnaby John James (Los Gatos, CA); Tatiana Kichkaylo (Mountain View, CA); Peter Kimball (Mountain View, CA); Christopher Thornton (Mountain View, CA)
Assignee: X Development LLC
H04N5/23299A01K29/00A01K61/13G06T7/50G06T2207/30232G06T2207/30242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,089,227
App. No.
16/785,252
Granted
Aug 10, 2021
Kind
B1
Abstract

A method for controlling a sensor subsystem, the method including receiving one or more metrics representing one or more characteristics of livestock, including one or more livestock objects, contained in an enclosure and monitored by one or more sensors coupled to a winch subsystem. The method further includes determining a position to move the one or more sensors based on the metrics and determining an instruction that includes information related to a movement of the one or more sensors. The method further includes sending the instruction to the winch subsystem to change the position of the one or more sensors.

Claims (151)

1. A method performed by a biomass monitoring system, the method comprising:

receiving one or more metrics representing one or more characteristics of livestock, including one or more livestock objects, contained in an enclosure and monitored by one or more sensors coupled to a winch subsystem,

wherein receiving the one or more metrics representing the one or more characteristics of the livestock includes determining a median distance from the one or more sensors to the livestock, the median distance corresponding to a median value of each distance from each of the one or more livestock objects to the one or more sensors;

determining that the median distance is greater than or equal to an upper threshold median distance;

based on determining that the median distance is greater than or equal to the upper threshold median distance, determining a position to move the one or more sensors;

determining an instruction that comprises information related to a movement of the one or more sensors, wherein the information of the instruction relates to moving the one or more sensors closer to the livestock; and

sending the instruction to the winch subsystem to change the position of the one or more sensors.

2. The method of claim 1 , wherein the one or more metrics includes a number of individual livestock object detections, a number of stereo matched livestock tracks, a median depth offset of the livestock, a median object track duration of the livestock, or a median livestock track angle.

3. The method of claim 1 , further comprising:

receiving sensor data from the one or more sensors; and

generating, using the sensor data, the one or more metrics.

4. The method of claim 1 , further comprising:

receiving, by the winch subsystem, the instructions; and

changing, by the winch subsystem, the position of the one or more sensors according to the instructions.

5. The method of claim 1 , further comprising:

determining an approximate number of livestock objects of the livestock;

determining that the approximate number of livestock objects is less than a threshold number of livestock objects; and

wherein the information of the instruction relates to a change in position of the one or more sensors from a first depth within the enclosure to a second depth within the enclosure.

6. The method of claim 1 , further comprising:

determining a first depth within the enclosure of the one or more sensors;

calculating a depth offset value being a difference between the first depth and a reference depth within the enclosure of the one or more sensors;

determining that the depth offset value is greater than or equal to a threshold depth offset value; and

wherein the information of the instruction relates to a change in position of the one or more sensors from the first depth within the enclosure to a second depth within the enclosure.

7. The method of claim 1 , further comprising:

determining a first depth within the enclosure of the one or more sensors;

calculating a depth offset value being a difference between the first depth and a reference depth within the enclosure of the one or more sensors;

determining that the depth offset value is greater than or equal to a threshold depth offset value; and

wherein the information of the instruction relates to a change in position of the one or more sensors from the first depth within the enclosure to a second depth within the enclosure.

8. The method of claim 1 , further comprising:

determining a median track angle of the livestock, the median track angle corresponding to a median value of one or more angles formed between a track of one or more livestock objects and a horizontal line;

determining that the median track angle of the livestock is greater than or equal to a threshold track offset value; and

wherein the information of the instruction relates to a change in an angle of the winch subsystem from an initial angle to an updated angle.

9. The method of claim 1 , further comprising:

determining a median distance from the one or more sensors to the livestock, the median distance corresponding to a median value of each distance from each of the one or more livestock objects to the one or more sensors; and

determining that the median distance is less than or equal to a lower threshold median distance;

wherein the information of the instruction relates to moving the one or more sensors farther from the livestock.

10. The method of claim 1 , further comprising generating, using the metrics, a belief matrix comprising a plurality of entries each representing a likelihood of livestock being at a certain location within the enclosure.

11. The method of claim 10 , wherein the belief matrix further comprises, for each of the likelihoods of the plurality of entries, a confidence score for the likelihood.

12. The method of claim 10 , wherein the information of the instruction relates to moving the one or more sensors to a particular location within the enclosure that corresponds to a particular entry of the belief matrix the particular location having the greatest likelihood of livestock being present at the particular location compared to other locations within the enclosure.

13. The method of claim 1 , wherein the one or more sensors include a camera, an IR sensor, a UV sensor, a temperature sensor, a pressure sensor, a hydrophone, a water current sensor, or a water quality sensor such as one that detects oxygen saturation or an amount of a dissolved solid.

14. The method of claim 1 , wherein the characteristics of the livestock include being hungry, sick, hurt, or dead.

15. A biomass monitoring system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

receiving one or more metrics representing one or more characteristics of livestock, including one or more livestock objects, contained in an enclosure and monitored by one or more sensors coupled to a winch subsystem,

wherein receiving the one or more metrics representing the one or more characteristics of the livestock includes determining a median distance from the one or more sensors to the livestock, the median distance corresponding to a median value of each distance from each of the one or more livestock objects to the one or more sensors;

determining that the median distance is greater than or equal to an upper threshold median distance;

based on determining that the median distance is greater than or equal to the upper threshold median distance, determining a position to move the one or more sensors;

determining an instruction that comprises information related to a movement of the one or more sensors, wherein the information of the instruction relates to moving the one or more sensors closer to the livestock; and

sending the instruction to the winch subsystem to change the position of the one or more sensors.

16. The system of claim 15 , wherein the one or more sensors include a camera, an IR sensor, a UV sensor, a temperature sensor, a pressure sensor, a hydrophone, a water current sensor, or a water quality sensor such as one that detects oxygen saturation or an amount of a dissolved solid.

17. The system of claim 15 , the operations further comprising:

receiving, by the winch subsystem, the instructions; and

changing, by the winch subsystem, the position of the one or more sensors according to the instructions.

18. The system of claim 15 , the operations further comprising:

receiving sensor data from the one or more sensors; and

generating, using the sensor data, the one or more metrics.

19. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving one or more metrics representing one or more characteristics of livestock, including one or more livestock objects, contained in an enclosure and monitored by one or more sensors coupled to a winch subsystem,

wherein receiving the one or more metrics representing the one or more characteristics of the livestock includes determining a median distance from the one or more sensors to the livestock, the median distance corresponding to a median value of each distance from each of the one or more livestock objects to the one or more sensors;

determining that the median distance is greater than or equal to an upper threshold median distance;

based on determining that the median distance is greater than or equal to the upper threshold median distance, determining a position to move the one or more sensors;

determining an instruction that comprises information related to a movement of the one or more sensors, wherein the information of the instruction relates to moving the one or more sensors closer to the livestock; and

sending the instruction to the winch subsystem to change the position of the one or more sensors.

20. The medium of claim 19 , wherein the one or more sensors include a camera, an IR sensor, a UV sensor, a temperature sensor, a pressure sensor, a hydrophone, a water current sensor, or a water quality sensor such as one that detects oxygen saturation or an amount of a dissolved solid.

21. The medium of claim 19 , the operations further comprising:

receiving, by the winch subsystem, the instructions; and

changing, by the winch subsystem, the position of the one or more sensors according to the instructions.

22. The medium of claim 19 , the operations further comprising:

receiving sensor data from the one or more sensors; and

generating, using the sensor data, the one or more metrics.

23. A method performed by a biomass monitoring system, the method comprising:

receiving one or more metrics representing one or more characteristics of livestock, including one or more livestock objects, contained in an enclosure and monitored by one or more sensors coupled to a winch subsystem,

wherein receiving the one or more metrics representing the one or more characteristics of the livestock includes determining a median distance from the one or more sensors to the livestock, the median distance corresponding to a median value of each distance from each of the one or more livestock objects to the one or more sensors;

based on determining that the median distance is less than or equal to the lower threshold median distance, determining a position to move the one or more sensors;

determining an instruction that comprises information related to a movement of the one or more sensors, wherein the information of the instruction relates to moving the one or more sensors farther from the livestock; and

sending the instruction to the winch subsystem to change the position of the one or more sensors.

24. The method of claim 23 , wherein the one or more sensors include a camera, an IR sensor, a UV sensor, a temperature sensor, a pressure sensor, a hydrophone, a water current sensor, or a water quality sensor such as one that detects oxygen saturation or an amount of a dissolved solid.

25. The method of claim 23 , further comprising:

receiving, by the winch subsystem, the instructions; and

changing, by the winch subsystem, the position of the one or more sensors according to the instructions.

26. The method of claim 23 , further comprising:

receiving sensor data from the one or more sensors; and

generating, using the sensor data, the one or more metrics.

27. A biomass monitoring system comprising:

receiving one or more metrics representing one or more characteristics of livestock, including one or more livestock objects, contained in an enclosure and monitored by one or more sensors coupled to a winch subsystem,

wherein receiving the one or more metrics representing the one or more characteristics of the livestock includes determining a median distance from the one or more sensors to the livestock, the median distance corresponding to a median value of each distance from each of the one or more livestock objects to the one or more sensors;

based on determining that the median distance is less than or equal to the lower threshold median distance, determining a position to move the one or more sensors;

determining an instruction that comprises information related to a movement of the one or more sensors, wherein the information of the instruction relates to moving the one or more sensors farther from the livestock; and

sending the instruction to the winch subsystem to change the position of the one or more sensors.

28. The system of claim 27 , wherein the one or more sensors include a camera, an IR sensor, a UV sensor, a temperature sensor, a pressure sensor, a hydrophone, a water current sensor, or a water quality sensor such as one that detects oxygen saturation or an amount of a dissolved solid.

29. The system of claim 27 , further comprising:

receiving, by the winch subsystem, the instructions; and

changing, by the winch subsystem, the position of the one or more sensors according to the instructions.

30. The system of claim 27 , the operations further comprising:

receiving sensor data from the one or more sensors; and

generating, using the sensor data, the one or more metrics.

31. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving one or more metrics representing one or more characteristics of livestock, including one or more livestock objects, contained in an enclosure and monitored by one or more sensors coupled to a winch subsystem,

wherein receiving the one or more metrics representing the one or more characteristics of the livestock includes determining a median distance from the one or more sensors to the livestock, the median distance corresponding to a median value of each distance from each of the one or more livestock objects to the one or more sensors;

based on determining that the median distance is less than or equal to the lower threshold median distance, determining a position to move the one or more sensors;

determining an instruction that comprises information related to a movement of the one or more sensors, wherein the information of the instruction relates to moving the one or more sensors farther from the livestock; and

sending the instruction to the winch subsystem to change the position of the one or more sensors.

32. The medium of claim 31 , wherein the one or more sensors include a camera, an IR sensor, a UV sensor, a temperature sensor, a pressure sensor, a hydrophone, a water current sensor, or a water quality sensor such as one that detects oxygen saturation or an amount of a dissolved solid.

33. The medium of claim 31 , further comprising:

receiving, by the winch subsystem, the instructions; and

changing, by the winch subsystem, the position of the one or more sensors according to the instructions.

34. The medium of claim 31 , the operations further comprising:

receiving sensor data from the one or more sensors; and

generating, using the sensor data, the one or more metrics.

35. A method performed by a biomass monitoring system, the method comprising:

receiving one or more metrics representing one or more characteristics of livestock, including one or more livestock objects, contained in an enclosure and monitored by one or more sensors coupled to a winch subsystem,

wherein receiving the one or more metrics representing the one or more characteristics of the livestock includes determining a median track angle of the livestock, the median track angle corresponding to a median value of one or more angles formed between a track of the one or more livestock objects and a horizontal line;

determining that the median track angle of the livestock is greater than or equal to a threshold track offset value;

based on determining that the median track angle of the livestock is greater than or equal to the threshold track offset value, determining a position to move the one or more sensors;

determining an instruction that comprises information related to a movement of the one or more sensors, wherein the information of the instruction relates to a change in an angle of the winch subsystem from an initial angle to an updated angle; and

sending the instruction to the winch subsystem to change the position of the one or more sensors.

36. The method of claim 35 , wherein the one or more sensors include a camera, an IR sensor, a UV sensor, a temperature sensor, a pressure sensor, a hydrophone, a water current sensor, or a water quality sensor such as one that detects oxygen saturation or an amount of a dissolved solid.

37. The method of claim 35 , further comprising:

receiving, by the winch subsystem, the instructions; and

changing, by the winch subsystem, the position of the one or more sensors according to the instructions.

38. The method of claim 35 , further comprising:

receiving sensor data from the one or more sensors; and

generating, using the sensor data, the one or more metrics.

39. A biomass monitoring system comprising:

receiving one or more metrics representing one or more characteristics of livestock, including one or more livestock objects, contained in an enclosure and monitored by one or more sensors coupled to a winch subsystem,

wherein receiving the one or more metrics representing the one or more characteristics of the livestock includes determining a median track angle of the livestock, the median track angle corresponding to a median value of one or more angles formed between a track of the one or more livestock objects and a horizontal line;

determining that the median track angle of the livestock is greater than or equal to a threshold track offset value;

based on determining that the median track angle of the livestock is greater than or equal to the threshold track offset value, determining a position to move the one or more sensors;

determining an instruction that comprises information related to a movement of the one or more sensors, wherein the information of the instruction relates to a change in an angle of the winch subsystem from an initial angle to an updated angle; and

sending the instruction to the winch subsystem to change the position of the one or more sensors.

40. The system of claim 39 , wherein the one or more sensors include a camera, an IR sensor, a UV sensor, a temperature sensor, a pressure sensor, a hydrophone, a water current sensor, or a water quality sensor such as one that detects oxygen saturation or an amount of a dissolved solid.

41. The system of claim 39 , the operations further comprising:

receiving, by the winch subsystem, the instructions; and

changing, by the winch subsystem, the position of the one or more sensors according to the instructions.

42. The system of claim 39 , the operations further comprising:

receiving sensor data from the one or more sensors; and

generating, using the sensor data, the one or more metrics.

43. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving one or more metrics representing one or more characteristics of livestock, including one or more livestock objects, contained in an enclosure and monitored by one or more sensors coupled to a winch subsystem,

wherein receiving the one or more metrics representing the one or more characteristics of the livestock includes determining a median track angle of the livestock, the median track angle corresponding to a median value of one or more angles formed between a track of the one or more livestock objects and a horizontal line;

determining that the median track angle of the livestock is greater than or equal to a threshold track offset value;

based on determining that the median track angle of the livestock is greater than or equal to the threshold track offset value, determining a position to move the one or more sensors;

determining an instruction that comprises information related to a movement of the one or more sensors, wherein the information of the instruction relates to a change in an angle of the winch subsystem from an initial angle to an updated angle; and

sending the instruction to the winch subsystem to change the position of the one or more sensors.

44. The medium of claim 43 , wherein the one or more sensors include a camera, an IR sensor, a UV sensor, a temperature sensor, a pressure sensor, a hydrophone, a water current sensor, or a water quality sensor such as one that detects oxygen saturation or an amount of a dissolved solid.

45. The medium of claim 43 , the operations further comprising:

receiving, by the winch subsystem, the instructions; and

changing, by the winch subsystem, the position of the one or more sensors according to the instructions.

46. The medium of claim 43 , the operations further comprising:

receiving sensor data from the one or more sensors; and

generating, using the sensor data, the one or more metrics.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2024
From: X DEVELOPMENT LLC
To: TIDALX AI INC.
Reel/Frame 068477/0306 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 20, 2020
From: JAMES, BARNABY JOHN; KICHKAYLO, TATIANA; KIMBALL, PETER; THORNTON, CHRISTOPHER
To: X DEVELOPMENT LLC
Reel/Frame 052444/0067 →
Cited By (2)
US 12,277,722 US 12,375,310