Precise book page boundary detection using deep machine learning model and image processing algorithms

1 . A method comprising:

receiving a digital image;

obtaining a plurality of outputs based on inputting the digital image to a convolutional neural segmentation network, the plurality of outputs comprising a first output and a second output, the first output for a first outer boundary of a left page of a two-page spread of a physical book, the second output for a second outer boundary of a right page of the two-page spread, each output including a thick pixel mask and x-offset and y-offset tensors storing distance values to nearest points on respective outer boundaries;

determining a first thin pixel mask for the first outer boundary based on the first output;

determining a second thin pixel mask for the second outer boundary based on the second output;

determining a first quadrilateral for the first outer boundary based on the first thin pixel mask;

determining a second quadrilateral for the second outer boundary based on the second thin pixel mask; and

presenting the first quadrilateral and the second quadrilateral for display in a graphical user interface.

2 . The method of claim 1 , wherein:

the first output comprises a first thick pixel mask, a first x-offset tensor, and a first y-offset tensor;

the second output comprises a second thick pixel mask, a second x-offset tensor, and a second y-offset tensor;

determining the first thin pixel mask is based on the first thick pixel mask, the first x-offset tensor, and the first y-offset tensor for the first outer boundary; and

determining the second thin pixel mask is based on the second thick pixel mask, the second x-offset tensor, and the second y-offset tensor.

3 . The method of claim 2 , wherein:

determining the first thin pixel mask is based on accumulating a first set of values in an accumulator tensor representing a grid of image pixels based on the first thick pixel mask, the first x-offset tensor, and the first y-offset tensor; and

determining the second thin pixel mask is based on accumulating a second set of values in an accumulator tensor representing a grid of image pixels based on the second thick pixel mask, the second x-offset tensor, and the second y-offset tensor.

4 . The method of claim 3 , wherein:

the digital image is a downsampled version of a source digital image, the downsampled version having an image resolution;

a grid of pixels represented by an accumulator tensor has width and height dimensions that are a scaling factor greater than the image resolution of the downsampled version;

accumulating the first set of values in an accumulator tensor is based on the scaling factor; and

accumulating the second set of values in an accumulator tensor is based on the scaling factor.

5 . The method of claim 1 , further comprising:

determining an outer boundary of the two-page spread;

determining a bind line search region of a digital image based on the outer boundary;

wherein the bind line search region is entirely within the outer boundary;

determining a set of one or more Hough lines within the bind line search region; and

determining a bind line of the two-page spread based on the set of one or more Hough lines.

6 . The method of claim 5 , wherein:

the set of one or more Hough lines comprises a plurality of Hough lines;

determining the bind line of the two-page spread is based on selecting a particular Hough line of the plurality of Hough lines; and

selecting the particular Hough line of the plurality of Hough lines is based on a distance of the particular Hough line from a center of the two-page spread, an angle of the particular Hough line relative to a vertical line of the two-page spread, or a strength of a gradient underneath the particular Hough line.

7 . The method of claim 1 , wherein the convolutional neural segmentation network is trained using a loss function that masks page boundary x-offset outputs and page boundary y-offset outputs with a ground truth page boundary pixel mask.

8 . A non-transitory computer-readable medium storing executable instructions which, when executed by a processing device, cause the processing device to perform operations comprising:

obtaining a digital image captured by a digital camera; wherein the digital image comprises a spread of a physical book;

inputting the digital image to a convolutional neural segmentation network to obtain a plurality of outputs; wherein the plurality of outputs comprise first output and second output, the first output for a first boundary of a left page of the spread, the second output for a second boundary of a right page of the spread, each output including a thick pixel mask and x-offset and y-offset tensors storing distance values to nearest points on respective outer boundaries;

determining a first thin pixel mask for the first boundary based on the first output;

determining a second thin pixel mask for the second boundary based on the second output;

determining, based on the first thin pixel mask and the second thin pixel mask, a first quadrilateral for the first boundary and a second quadrilateral for the second boundary; and

displaying at least one of the first quadrilateral for the first boundary or the second quadrilateral for the second boundary in a graphical user interface.

9 . The non-transitory computer-readable medium of claim 8 , wherein:

the first output comprises a first thick pixel mask, a first x-offset tensor, and a first y-offset tensor;

the second output comprises a second thick pixel mask, a second x-offset tensor, and a second y-offset tensor;

determining the first thin pixel mask is based on the first thick pixel mask, the first x-offset tensor, and the first y-offset tensor; and

determining the second thin pixel mask is based on the second thick pixel mask, the second x-offset tensor, and the second y-offset tensor.

10 . The non-transitory computer-readable medium of claim 9 , wherein:

determining the first thin pixel mask is based on accumulating a first set of values in an accumulator tensor representing a grid of image pixels based on the first thick pixel mask, the first x-offset tensor, and the first y-offset tensor; and

determining the second thin pixel mask is based on accumulating a second set of values in an accumulator tensor representing a grid of image pixels based on the second thick pixel mask, the second x-offset tensor, and the second y-offset tensor.

11 . The non-transitory computer-readable medium of claim 10 , wherein:

the digital image is a downsampled version of a source digital image, the downsampled version having an image resolution;

a grid of pixels represented by an accumulator tensor has width and height dimensions that are a scaling factor greater than the image resolution of the downsampled version;

accumulating the first set of values in an accumulator tensor is based on the scaling factor; and

accumulating the second set of values in an accumulator tensor is based on the scaling factor.

12 . The non-transitory computer-readable medium of claim 8 , further storing executable instructions which, when executed by the processing device, cause the processing device to perform operations comprising:

determining an outer boundary of the spread;

determining a bind line search region of a digital image based on the outer boundary;

wherein the bind line search region is entirely within the outer boundary;

determining a set of one or more Hough lines within the bind line search region; and

determining a bind line of the spread based on the set of one or more Hough lines.

13 . The non-transitory computer-readable medium of claim 12 , wherein:

the set of one or more Hough lines comprises a plurality of Hough lines;

determining the bind line of the spread is based on selecting a particular Hough line of the plurality of Hough lines; and

selecting the particular Hough line of the plurality of Hough lines is based on a distance of the particular Hough line from a center of the spread, an angle of the particular Hough line relative to a vertical line of the spread, or a strength of a gradient underneath the particular Hough line.

14 . A system comprising:

a set of one or more memory components; and

a set of one or more processing devices coupled to the set of one or more memory components, the set of one or more processing devices to perform a set of operations comprising:

receiving a digital image;

obtaining, based on inputting the digital image to a convolutional neural segmentation network, a plurality of outputs comprising first output and second output, the first output for a first boundary of a left page of a two-page spread of a physical book, the second output for a second boundary of a right page of the two-page spread, each output including a thick pixel mask and x-offset and y-offset tensors storing distance values to nearest points on respective outer boundaries;

determining a first thin pixel mask for the first boundary based on the first output;

determining a second thin pixel mask for the second boundary based on the second output;

determining a first quadrilateral for the first boundary based on the first thin pixel mask;

determining a second quadrilateral for the second boundary based on the second thin pixel mask; and

presenting the first quadrilateral and the second quadrilateral for display in a graphical user interface.

15 . The system of claim 14 , wherein:

the first output comprises a first thick pixel mask, a first x-offset tensor, and a first y-offset tensor;

the second output comprises a second thick pixel mask, a second x-offset tensor, and a second y-offset tensor;

determining the first thin pixel mask is based on the first thick pixel mask, the first x-offset tensor, and the first y-offset tensor; and

determining the second thin pixel mask is based on the second thick pixel mask, the second x-offset tensor, and the second y-offset tensor.

16 . The system of claim 15 , wherein:

determining the first thin pixel mask is based on accumulating a first set of values in an accumulator tensor representing a grid of image pixels based on the first thick pixel mask, the first x-offset tensor, and the first y-offset tensor; and

determining the second thin pixel mask is based on accumulating a second set of values in an accumulator tensor representing a grid of image pixels based on the second thick pixel mask, the second x-offset tensor, and the second y-offset tensor.

17 . The system of claim 16 , wherein:

the digital image is a downsampled version of a source digital image, the downsampled version having an image resolution;

a grid of pixels represented by an accumulator tensor has width and height dimensions that are a scaling factor greater than the image resolution of the downsampled version;

accumulating the first set of values in an accumulator tensor is based on the scaling factor; and

accumulating the second set of values in an accumulator tensor is based on the scaling factor.

18 . The system of claim 14 , the set of one or more processing devices to further perform operations comprising:

determining an outer boundary of the two-page spread;

determining a bind line search region of a digital image based on the outer boundary;

wherein the bind line search region is entirely within the outer boundary;

determining a set of one or more Hough lines within the bind line search region; and

determining a bind line of the two-page spread based on the set of one or more Hough lines.

19 . The system of claim 18 , wherein:

the set of one or more Hough lines comprises a plurality of Hough lines;

determining the bind line of the two-page spread is based on selecting a particular Hough line of the plurality of Hough lines; and

selecting the particular Hough line of the plurality of Hough lines is based on a distance of the particular Hough line from a center of the two-page spread, an angle of the particular Hough line relative to a vertical line of the two-page spread, or a strength of a gradient underneath the particular Hough line.

20 . The system of claim 18 , wherein the convolutional neural segmentation network is trained using a loss function that masks page boundary x-offset outputs and page boundary y-offset outputs with a ground truth page boundary pixel mask.