Fish biomass, shape, size, or health determination
Methods, systems, and apparatuses, including computer programs encoded on a computer-readable storage medium for estimating the shape, size, mass, and health of fish are described. A pair of stereo cameras may be utilized to obtain off-axis images of fish in a defined area. The images may be processed, enhanced, and combined. Object detection may be used to detect and track a fish in images. A pose estimator may be used to determine key points and features of the detected fish. Based on the key points, a model of the fish is generated that provides an estimate of the size and shape of the fish. A regression model or neural network model can be applied to the fish model to determine characteristics of the fish.
1. A computer-implemented method comprising:
obtaining, by one or more processors, one or more images of a fish;
determining one or more key points associated with one or more features of the fish in the one or more images;
determining, using a model of the fish that is based on distances between the one or more key points, a value for a characteristic of the fish, wherein the characteristic includes any of biomass, shape, size, or health; and
outputting a representation of the characteristic of the fish for display or storage at a device connected to the one or more processors.
2. The computer-implemented method of claim 1 , wherein
determining the value for the characteristic further comprises:
determining any of the following health conditions: shortened abdomen, shortened tail, scoliosis, lordosis, kyphosis, deformed upper jaw, deformed lower jaw, shortened operculum, runting or cardiomyopathy syndrome (CMS).
3. The computer-implemented method of claim 1 , comprising:
generating a single image from the one or more images;
generating a depth map for the single image; and
identifying the fish and one or more regions of interest in the single image by performing object detection using a recurrent convolutional neural network.
4. The computer-implemented method of claim 3 , wherein:
the one or more key points associated with the one or more features of the fish are determined for each of the one or more regions of interest using pose estimation;
the one or more images are obtained using one or more image acquisition devices; and
the one or more images include an image from one image acquisition device and another image from a different image acquisition device.
5. The computer-implemented method of claim 1 , wherein:
the determined one or more key points include one or more two-dimensional key points; and
generating the model of the fish comprises:
generating a 3D model of the fish.
6. The computer-implemented method of claim 5 , wherein
generating a 3D model of the fish comprises:
determining three-dimensional key points for the fish by using the determined one or more two-dimensional key points and a depth map.
7. The computer-implemented method of claim 1 , wherein:
generating the model of the fish comprises:
determining a truss network comprised of length values, wherein the length values indicate distances between key points.
8. The computer-implemented method of claim 1 , wherein
determining the value for the characteristic of the fish using the model of the fish comprises:
applying a linear regression model to the model of the fish.
9. The computer-implemented method of claim 1 , comprising:
obtaining one or more secondary images of the fish; and
determining the value of the characteristic of the fish based on the obtained one or more secondary images of the fish and the model of the fish.
10. The computer-implemented method of claim 1 , comprising:
training a neural network classifier using a pose estimation model to predict likely key points of the fish.
11. A system comprising:
one or more computing devices and one or more storage devices storing instructions which when executed by the one or more computing devices, cause the one or more computing devices to perform operations comprising:
obtaining one or more images of a fish;
determining one or more key points associated with one or more features of the fish in the one or more images;
determining, using a model of the fish that is based on distances between the one or more key points, a value for a characteristic of the fish, wherein the characteristic includes any of biomass, shape, size, or health; and
outputting a representation of the characteristic of the fish for display or storage at a device connected to the one or more computing devices.
12. The system of claim 11 , wherein determining the value for the characteristic further comprises:
determining any of the following health conditions: shortened abdomen, shortened tail, scoliosis, lordosis, kyphosis, deformed upper jaw, deformed lower jaw, shortened operculum, runting or cardiomyopathy syndrome (CMS).
13. The system of claim 11 , wherein the operations comprise:
generating a single image from the one or more images;
generating a depth map for the single image; and
identifying the fish and one or more regions of interest in the single image by performing object detection using a recurrent convolutional neural network.
14. The system of claim 13 , wherein:
the one or more key points associated with the one or more features of the fish are determined for each of the one or more regions of interest using pose estimation;
the one or more images are obtained using one or more image acquisition devices; and
the one or more images include an image from one image acquisition device and another image from a different image acquisition device.
15. The system of claim 11 , wherein:
the determined one or more key points include one or more two-dimensional key points; and
generating the model of the fish comprises:
generating a 3D model of the fish.
16. The system of claim 15 , wherein generating a 3D model of the fish comprises:
determining three-dimensional key points for the fish by using the determined one or more two-dimensional key points and a depth map.
17. The system of claim 11 , wherein:
generating the model of the fish comprises:
determining a truss network comprised of length values, wherein the length values indicate distances between key points.
18. The system of claim 11 , wherein determining the value for the characteristic of the fish using the model of the fish comprises:
applying a linear regression model to the model of the fish.
19. The system of claim 11 , wherein the operations comprise:
obtaining one or more secondary images of the fish; and
determining the value of the characteristic of the fish based on the obtained one or more secondary images of the fish.
20. One or more non-transitory computer-readable storage media comprising instructions, which, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising:
obtaining one or more images of a fish;
determining one or more key points associated with one or more features of the fish in the one or more images;
determining, using a model of the fish that is based on distances between the one or more key points, a value for a characteristic of the fish, wherein the characteristic includes any of biomass, shape, size, or health; and
outputting a representation of the characteristic of the fish for display or storage at a device connected to the one or more computing devices.