Fish biomass, shape, and size determination
Methods, systems, and apparatuses, including computer programs encoded on a computer-readable storage medium for estimating the shape, size, and mass of fish are described. A pair of stereo cameras may be utilized to obtain right and left images of fish in a defined area. The right and left images may be processed, enhanced, and combined. Object detection may be used to detect and track a fish in images. A pose estimator may be used to determine key points and features of the detected fish. Based on the key points, a three-dimensional (3-D) model of the fish is generated that provides an estimate of the size and shape of the fish. A regression model or neural network model can be applied to the 3-D model to determine a likely weight of the fish.
1. A computer-implemented method comprising:
determining, by a camera system, that an image that was generated by the camera system is of a fish that is at least partially occluded;
determining, by the camera system, a respective position of each of one or more key points of the fish that are visible in the image;
determining, by the camera system, a respective position of each of one or more key points of the fish that are occluded within the image based at least on the determined respective position of each of the one or more key points of the fish that are visible in the image; and
generating, by the camera system, a weigh estimate of the fish that is at least partially occluded based on the determined respective position of each of the one or more key points of the fish that are occluded within the image; and
providing, by the camera system, the weight estimate of the fish for output.
2. The method of claim 1 , wherein the camera system is an underwater stereo camera system.
3. The method of claim 1 , wherein the one or more key points of the fish that are occluded within the image include an eye, a nostril, or an operculum.
4. The method of claim 1 , wherein the respective position of each of the one or more key points of the fish that are occluded within the image are determined using a probability model.
5. The method of claim 1 , wherein the respective position of each of the one or more key points of the fish that are occluded within the image are determined using data that indicates likely positions of key points of the fish.
6. The method of claim 1 , wherein generating the weight estimate comprises generating a 3D model of the fish based on the positions of the key points.
7. The method of claim 1 , comprising ranking 3D models of the fish that are generated based on the positions of the key points.
8. A system comprising:
one or more processing devices;
one or more machine-readable storage devices for storing instructions that are executable by the one or more processing devices to perform operations comprising:
determining, by a camera system, that an image that was generated by the camera system is of a fish that is at least partially occluded;
determining, by the camera system, a respective position of each of one or more key points of the fish that are visible in the image;
determining, by the camera system, a respective position of each of one or more key points of the fish that are occluded within the image based at least on the determined respective position of each of the one or more key points of the fish that are visible in the image; and
generating, by the camera system, a weigh estimate of the fish that is at least partially occluded based on the determined respective position of each of the one or more key points of the fish that are occluded within the image; and
providing, by the camera system, the weight estimate of the fish for output.
9. The system of claim 8 , wherein the camera system is an underwater stereo camera system.
10. The system of claim 8 , wherein the one or more key points of the fish that are occluded within the image include an eye, a nostril, or an operculum.
11. The system of claim 8 , wherein the respective position of each of the one or more key points of the fish that are occluded within the image are determined using a probability model.
12. The system of claim 8 , wherein the respective position of each of the one or more key points of the fish that are occluded within the image are determined using data that indicates likely positions of key points of the fish.
13. The system of claim 8 , wherein generating the weight estimate comprises generating a 3D model of the fish based on the positions of the key points.
14. The system of claim 8 , wherein the operations comprise ranking 3D models of the fish that are generated based on the positions of the key points.
15. A non-transitory machine-readable storage medium for storing instructions that are executable by the one or more processing devices to perform operations comprising:
determining, by a camera system, that an image that was generated by the camera system is of a fish that is at least partially occluded;
determining, by the camera system, a respective position of each of one or more key points of the fish that are visible in the image;
determining, by the camera system, a respective position of each of one or more key points of the fish that are occluded within the image based at least on the determined respective position of each of the one or more key points of the fish that are visible in the image; and
generating, by the camera system, a weigh estimate of the fish that is at least partially occluded based on the determined respective position of each of the one or more key points of the fish that are occluded within the image; and
providing, by the camera system, the weight estimate of the fish for output.
16. The medium of claim 15 , wherein the camera system is an underwater stereo camera system.
17. The medium of claim 15 , wherein the one or more key points of the fish that are occluded within the image include an eye, a nostril, or an operculum.
18. The medium of claim 15 , wherein the respective position of each of the one or more key points of the fish that are occluded within the image are determined using a probability model.
19. The medium of claim 15 , wherein the respective position of each of the one or more key points of the fish that are occluded within the image are determined using data that indicates likely positions of key points of the fish.
20. The medium of claim 15 , wherein generating the weight estimate comprises generating a 3D model of the fish based on the positions of the key points.