Systems and methods for identifying and segmenting objects from images
Systems and methods for identifying and segmenting objects from images include a preprocessing module configured to adjust a size of a source image; a region-proposal module configured to propose one or more regions of interest in the size-adjusted source image; and a prediction module configured to predict a classification, bounding box coordinates, and mask. Such systems and methods may utilize end-to-end training of the modules using adversarial loss, facilitating the use of a small training set, and can be configured to process historical documents, such as large images comprising text. The preprocessing module within said systems and methods can utilize a conventional image scaler in tandem with a custom image scaler to provide a resized image suitable for GPU processing, and the region-proposal module can utilize a region-proposal network from a single-stage detection model in tandem with a two-stage detection model paradigm to capture substantially all particles in an image.
1. A system for identifying and segmenting objects from images, the system comprising:
one or more memory devices storing a preprocessing module, a region-proposal module, and a prediction module; and
at least one processor coupled to the one or more memory devices, the at least one processor configured to perform operations comprising:
generating, utilizing the preprocessing module, a resized historical document image from a source historical document image;
determining, utilizing the region-proposal module, one or more regions of interest in the resized historical document image; and
generating, utilizing the prediction module to process pixels of the resized historical document image indicated by the one or more regions of interest, a predicted object defined by a classification and a set of bounding box coordinates within the resized historical document image.
2. The system of claim 1 , wherein:
the preprocessing module comprises a first image resizing model that applies a first resizing algorithm and a second image resizing model that applies a second resizing algorithm; and
generating the resized historical document image comprises utilizing the first resizing algorithm and the second resizing algorithm to resize the source historical document image.
3. The system of claim 2 , wherein the second resizing algorithm comprises a convolutional neural network that generates a filter corresponding to the source historical document image.
4. The system of claim 3 , wherein generating the resized historical document image comprises:
generating an intermediate resized historical document image utilizing the first resizing algorithm; and
combining the filter and the intermediate resized historical document image to generate the resized historical document image.
5. The system images of claim 1 , wherein the preprocessing module further comprises an image feature extractor configured to extract a feature map from the resized historical document image.
6. The system of claim 1 , wherein the region-proposal module comprises a first region-proposal algorithm for a two-stage region proposal neural network and a second region-proposal algorithm adapted from a single-stage region proposal neural network.
7. The system of claim 6 , wherein the first region-proposal algorithm is a region proposal network adapted from a region-based convolutional neural network (RCNN) algorithm and the second region-proposal algorithm is a region proposal network adapted from a you only look once (YOLO) based algorithm.
8. The system of claim 6 , wherein determining the one or more regions of interest comprises utilizing the first region-proposal algorithm and the second region-proposal algorithm in parallel.
9. The system of claim 1 , wherein generating the predicted object comprises segmenting objects within the resized historical document image utilizing the prediction module.
10. The system of claim 1 , wherein the prediction module comprises a classifier, a bounding box regressor, and a mask predictor.
11. The system of claim 1 , wherein the preprocessing module comprises a convolutional neural network having a kernel size through at least one layer of at least 7×7.
12. The system of claim 11 , wherein the convolutional neural network comprises a stride through at least one layer of at least 5.
13. The system of claim 12 , wherein a stride size of the convolutional neural network is less than a kernel size of the convolutional neural network.
14. A computer-implemented method comprising:
generating, utilizing a preprocessing module, a resized historical document image from a source historical document image;
determining, utilizing a region-proposal module comprising a combination of two region proposal models, one or more regions of interest in the resized historical document image; and
generating, utilizing a prediction module to process pixels of the resized historical document image indicated by the one or more regions of interest, a predicted object defined by a classification and a set of bounding box coordinates within the resized historical document image.
15. The computer-implemented method of claim 14 , wherein generating the resized historical document image comprises utilizing a first resizing algorithm and a second resizing algorithm of the preprocessing module to resize the source historical document image.
16. The computer-implemented method of claim 14 , wherein generating the predicted object comprises segmenting objects within the resized historical document image utilizing the prediction module.
17. A non-transitory computer readable medium storing instructions which, when executed by at least one processor, cause the at least one processor to:
generate, utilizing a preprocessing module comprising a learned image scaler, a resized historical document image from a source historical document image;
determine, utilizing a region-proposal module, one or more regions of interest in the resized historical document image; and
generate, utilizing a prediction module to process pixels of the resized historical document image indicated by the one or more regions of interest, a predicted object defined by a classification and a set of bounding box coordinates within the resized historical document image.
18. The non-transitory computer readable medium of claim 17 , further comprising instructions which, when executed by the at least one processor, cause the at least one processor to generate a region proposal within resized historical document image by utilizing a region-proposal neural network as part of the region-proposal module.
19. The non-transitory computer readable medium of claim 18 , further comprising instructions which, when executed by the at least one processor, cause the at least one processor to:
generate a bounding box coordinate utilizing a bounding box regressor;
generate an image mask for a bounding box defined by the bounding box coordinate utilizing a mask predictor; and
generate a segmented historical document image reflecting the image mask.
20. The non-transitory computer readable medium of claim 18 , further comprising instructions which, when executed by the at least one processor, cause the at least one processor to generate the resized historical document image by utilizing a first resizing algorithm and a second resizing algorithm of the preprocessing module to resize the source historical document image.