Automatic detection, counting, and measurement of lumber boards using a handheld device
An image processing system receives an image depicting a bundle of boards. The bundle of boards has a front face that is perpendicular to a long axis of boards and the image is captured at an angle relative to the long axis. The image processing system applies a homographic transformation to estimate a frontal view of the front face and identifies a plurality of divisions between rows in the estimate. For each adjacent pair of the plurality of divisions between rows, a plurality of vertical divisions is identified. The image processing system identifies a set of bounding boxes defined by pairs of adjacent divisions between rows and pairs of adjacent vertical divisions. The image processing system may filter and/or merge some bounding boxes to better match the bounding boxes to individual boards. Based on the bounding boxes, the image processing system determines the number of boards in the bundle.
1. A computer-implemented method for an image processing system to analyze a bundle of boards, the bundle of boards having a front face that is perpendicular to a long axis of boards in the bundle, the method comprising:
receiving an image, a portion of the image depicting the bundle of boards, the image captured at an angle relative to the long axis of boards in the bundle;
applying a homographic transformation to the image to estimate a frontal view of the front face, the frontal view being an appearance of the front face if viewed from a perspective parallel to the long axis of boards in the bundle;
identifying a set of bounding boxes using a neural network, wherein each bounding box represents the front face of a board within the bundle of boards; and
determining a number of boards in the bundle based on the set of bounding boxes.
2. The computer-implemented method of claim 1 , wherein the homographic transformation is based on corners of the bundle of boards identified using a second convolutional neural network.
3. The computer-implemented method of claim 1 , wherein the neural network was trained using training images, the training images depicting bundles of boards and labeled with bounding boxes indicating locations of front faces of boards in the training images.
4. The computer-implemented method of claim 3 , wherein the bounding boxes labeling the bundles of boards in the images on which the neural network trains are downsized from their original size.
5. The computer-implemented method of claim 1 , wherein the neural network comprises a first convolutional neural network that generates a segmentation map representing boards and a second convolutional neural network that generates a segmentation map representing centers of boards.
6. The computer-implemented method of claim 5 , wherein at least one of the first convolutional neural network and the second convolutional neural network is recomputed as a combination between itself and the other convolutional neural network.
7. The computer-implemented method of claim 1 , wherein the received image comprises data according to a red, green, and blue (RGB) color model.
8. A non-transitory computer-readable storage medium storing computer program instructions executable by a processor to perform operations to analyze a bundle of boards, the bundle of boards having a front face that is perpendicular to a long axis of boards in the bundle, the operations comprising:
receiving an image, a portion of the image depicting the bundle of boards, the image captured at an angle relative to the long axis of boards in the bundle;
applying a homographic transformation to the image to estimate a frontal view of the front face, the frontal view being an appearance of the front face if viewed from a perspective parallel to the long axis of boards in the bundle;
identifying a set of bounding boxes using a neural network, wherein each bounding box represents the front face of a board within the bundle of boards; and
determining a number of boards in the bundle based on the set of bounding boxes.
9. The non-transitory computer-readable storage medium of claim 8 , wherein the homographic transformation is based on corners of the bundle of boards identified using a second convolutional neural network.
10. The non-transitory computer-readable storage medium of claim 8 , wherein the neural network was trained using training images, the training images depicting bundles of boards and labeled with bounding boxes indicating locations of front faces of boards in the training images.
11. The non-transitory computer-readable storage medium of claim 10 , wherein the bounding boxes labeling the bundles of boards in the images on which the neural network trains are downsized from their original size.
12. The non-transitory computer-readable storage medium of claim 8 , wherein the neural network comprises a first convolutional neural network that generates a segmentation map representing boards and a second convolutional neural network that generates a segmentation map representing centers of boards.
13. The non-transitory computer-readable storage medium of claim 12 , wherein at least one of the first convolutional neural network and the second convolutional neural network is recomputed as a combination between itself and the other convolutional neural network.
14. The non-transitory computer-readable storage medium of claim 8 , wherein the received image comprises data according to a red, green, and blue (RGB) color model.
15. A system, comprising:
a processor for executing computer program instructions; and
a non-transitory computer-readable storage medium storing computer program instructions executable by the processor to perform operations to analyze a bundle of boards, the bundle of boards having a front face that is perpendicular to a long axis of boards in the bundle, the operations comprising:
receiving an image, a portion of the image depicting the bundle of boards, the image captured at an angle relative to the long axis of boards in the bundle;
applying a homographic transformation to the image to estimate a frontal view of the front face, the frontal view being an appearance of the front face if viewed from a perspective parallel to the long axis of boards in the bundle;
identifying a set of bounding boxes using a neural network, wherein each bounding box represents the front face of a board within the bundle of boards; and
determining a number of boards in the bundle based on the set of bounding boxes.
16. The system of claim 15 , wherein the homographic transformation is based on corners of the bundle of boards identified using a second convolutional neural network.
17. The system of claim 15 , wherein the neural network was trained using training images, the training images depicting bundles of boards and labeled with bounding boxes indicating locations of front faces of boards in the training images.
18. The system of claim 17 , wherein the bounding boxes labeling the bundles of boards in the images on which the neural network trains are downsized from their original size.
19. The system of claim 15 , wherein the neural network comprises a first convolutional neural network that generates a segmentation map representing boards and a second convolutional neural network that generates a segmentation map representing centers of boards.
20. The system of claim 19 , wherein at least one of the first convolutional neural network and the second convolutional neural network is recomputed as a combination between itself and the other convolutional neural network.