Systems and methods for identifying and segmenting objects from images
Systems and methods for identifying and segmenting objects from images include a preprocessing module configured to adjust a size of a source image; a region-proposal module configured to propose one or more regions of interest in the size-adjusted source image; and a prediction module configured to predict a classification, bounding box coordinates, and mask. Such systems and methods may utilize end-to-end training of the modules using adversarial loss, facilitating the use of a small training set, and can be configured to process historical documents, such as large images comprising text. The preprocessing module within the systems and methods can utilize a conventional image scaler in tandem with a custom image scaler to provide a resized image suitable for GPU processing, and the region-proposal module can utilize a region-proposal network from a single-stage detection model in tandem with a two-stage detection model paradigm to capture substantially all particles in an image.
1. A system for identifying and segmenting objects from images, the system comprising:
one or more memory devices storing a region-proposal module; and
a processor coupled to the one or more memory devices, the processor configured to perform operations comprising:
receiving an input document image, the input document image comprising a plurality of segmentable objects, at least one of the plurality of segmentable objects being boundable by bounding-box coordinates;
generating, using a region-proposal module, a plurality of bounding-box proposals for the plurality of segmentable objects in the received input document image, the region-proposal module comprising:
a first region-proposal model comprising a two-stage detector, and
a second region-proposal model comprising a single-stage detector;
wherein outputs of a combination of the first and second region-proposal models, the outputs corresponding to the segmentable objects in the received input document image, form the plurality of bounding-box proposals.
2. The system of claim 1 , further comprising:
a predictions module stored on the one or more memory devices; and
the processor being configured to perform operations further comprising:
generating, using a bounding-box regressor of the predictions module, bounding-box coordinates for the plurality of segmentable objects in the input document image based on the bounding-box proposals; and
outputting a segmented document image including the input document image and a plurality of bounding boxes corresponding to the plurality of segmentable objects.
3. The system of claim 2 , further comprising:
a proposal-extraction module stored on the one or more memory devices; and
the processor being configured to perform operations further comprising:
extracting features from each bounding-box proposal using the proposal extraction module; and
generating the bounding-box coordinates based on the extracted features using the bounding-box regressor of the predictions module.
4. The system of claim 3 , the processor being configured to perform operations further comprising:
generating a classification for each of the plurality of segmentable objects in the received input document image using a trained classifier of the predictions module.
5. The system of claim 1 , wherein the first and second region-proposal models receive feature maps corresponding to the received input document image.
6. The system of claim 5 , further comprising a proposal-extraction module stored on the one or more memory devices, wherein the region-proposal module and the proposal-extraction module are configured to receive the feature maps.
7. The system of claim 1 , wherein the region-proposal module is configured to generate x and y offsets for the bounding-box proposals as a function of a height and width dimension of the received input document image.
8. A computer-implemented method comprising:
receiving an input document image, the input document image comprising a plurality of segmentable objects, at least one of the plurality of segmentable objects being boundable by bounding-box coordinates;
generating, using a region-proposal module, a plurality of bounding-box proposals for the plurality of segmentable objects in the received input document image, the region-proposal module comprising:
a first region-proposal model comprising a two-stage detector, and
a second region-proposal model comprising a single-stage detector;
wherein outputs of a combination of the first and second region-proposal models, the outputs corresponding to the segmentable objects in the received input document image, form the plurality of bounding-box proposals.
9. The computer-implemented method of claim 8 , further comprising:
generating, using a bounding-box regressor of a predictions module, bounding-box coordinates for the plurality of segmentable objects in the input document image based on the bounding-box proposals; and
outputting a segmented document image including the input document image and a plurality of bounding boxes corresponding to the plurality of segmentable objects.
10. The computer-implemented method of claim 9 , further comprising:
extracting features from each bounding-box proposal using a proposal-extraction module; and
generating the bounding-box coordinates based on the extracted features using the bounding-box regressor of the predictions module.
11. The computer-implemented method of claim 10 , further comprising:
generating a classification for each of the plurality of segmentable objects in the received input document image using a trained classifier of the predictions module.
12. The computer-implemented method of claim 8 , wherein the first and second region-proposal models receive feature maps corresponding to the received input document image.
13. The computer-implemented method of claim 12 , wherein the region-proposal module and a proposal-extraction module are configured to receive the feature maps.
14. A non-transitory computer-readable medium storing instructions which, when executed by at least one processor, cause the at least one processor to:
receive an input document image, the input document image comprising a plurality of segmentable objects, at least one of the plurality of segmentable objects being boundable by bounding-box coordinates;
generate, using a region-proposal module, a plurality of bounding-box proposals for the plurality of segmentable objects in the received input document image, the region-proposal module comprising:
a first region-proposal model comprising a two-stage detector, and
a second region-proposal model comprising a single-stage detector;
wherein outputs of a combination of the first and second region-proposal models, the outputs corresponding to the segmentable objects in the received input document image, form the plurality of bounding-box proposals.
15. The non-transitory computer-readable medium of claim 14 , further comprising instructions which, when executed by the at least one processor, cause the at least one processor to:
generate, using a bounding-box regressor of a predictions module, bounding-box coordinates for the plurality of segmentable objects in the input document image based on the bounding-box proposals; and
output a segmented document image including the input document image and a plurality of bounding boxes corresponding to the plurality of segmentable objects.
16. The non-transitory computer-readable medium of claim 15 , further comprising instructions which, when executed by the at least one processor, cause the at least one processor to:
extract features from each bounding-box proposal using a proposal-extraction module; and
generate the bounding-box coordinates based on the extracted features using the bounding-box regressor of the predictions module.
17. The non-transitory computer-readable medium of claim 16 , further comprising instructions which, when executed by the at least one processor, cause the at least one processor to:
generate a classification for each of the plurality of segmentable objects in the received input document image using a trained classifier of the predictions module.
18. The non-transitory computer-readable medium of claim 14 , wherein the first and second region-proposal models receive feature maps corresponding to the received input document image.
19. The non-transitory computer-readable medium of claim 18 , wherein the region-proposal module and a proposal-extraction module are configured to receive the feature maps.
20. The non-transitory computer-readable medium of claim 14 , wherein the region-proposal module is configured to generate x and y offsets for the bounding-box proposals as a function of a height and width dimension of the received input document image.