Granted Patent B2

US 10,002,291 · App. 15/164,074 · Granted Jun 19, 2018

Inventors: Safwan R. Wshah (Webster, NY); Martin S. Maltz (Rochester, NY); Dennis L. Venable (Marion, NY); Saurabh Prabhat (Webster, NY)

Assignee: Xerox Corporation

G06K9/00449G06F3/04812G06F3/04842G06F17/243G06K9/00463

Abstract

Claims (90)

1. A method of identifying one or more fillable fields of an electronic form, the system comprising:

receiving, by an electronic device, an electronic form, wherein the electronic form comprises an image of a document;

identifying, by an electronic device, one or more fillable field candidates of the electronic form;

determining, for each fillable field candidate, whether the fillable field candidate is a fillable field by:

identifying one or more box candidates from the fillable field candidates, and

for one or more of the box candidates:

generating an out-border for the box candidate, wherein the out-border is a rectangular zone that surrounds an outer portion of the box candidate,

generating an in-border for the box candidate, wherein the in-border is a rectangular zone that encompasses at least a portion of the box candidate,

determining a histogram of pixels of the box candidate between the in-border and the out-border,

determining a ratio of black pixels to white pixels of the histogram, and

in response to the ratio exceeding a threshold value, determining that the box candidate is a fillable field, otherwise, determining that the box candidate is not a fillable field; and

updating metadata associated with the electronic form by applying, by the electronic device, a sequencing framework to only the fillable fields by:

obtaining position information for each fillable field, wherein the position information indicates a position of the fillable field on the document as displayed via a display device,

sorting the fillable fields based on the position information to form a sequence of fillable fields,

determining a designator to each fillable field, wherein the designator indicates a position of a corresponding fillable field in the sequence, and

storing the designator in a data store such that it is associated with the corresponding fillable field.

2. The method of claim 1 , wherein identifying the one or more box candidates from the fillable field candidates comprises one or more of the following:

using a seed fill algorithm to identify the one or more box candidates from the fillable field candidates; or

using a flood fill algorithm to identify the one or more box candidates from the fillable field candidates.

3. The method of claim 1 , further comprising:

in response to determining that the box candidate is a fillable field, determining whether the box candidate is a checkbox; and

in response to determining that the box candidate is a checkbox, adding the box candidate to a checkbox list, otherwise, adding the box candidate to a box list.

4. The method of claim 1 , wherein determining whether the fillable field candidate is a fillable field comprises determining whether the fillable field candidate is a line by:

obtaining dimensions for the fillable field candidate, wherein the dimensions includes a height value and a length value; and

in response to the height value not exceeding a first threshold value and the length value exceeding a second threshold value, identifying the fillable field candidate as a fillable field.

5. The method of claim 1 , wherein determining whether the fillable field candidate is a fillable field comprises determining whether the fillable field candidate is a line by:

obtaining dimensions for the fillable field candidate, wherein the dimensions includes a height value and a length value; and

in response to the length value not exceeding a first threshold value and the height value exceeding a second threshold value, identifying the fillable field candidate as a fillable field.

6. The method of claim 1 , wherein sorting the fillable fields based on the position information to form a sequence of fillable fields comprises sorting the fillable fields from a top of the document to a bottom of the document, and from a left side of the document to a right side of the document based on the position information.

7. The method of claim 1 , further comprising:

receiving a selection of a fillable field from a user, and causing a cursor to be displayed on a display device at the selected fillable field;

receiving an indication from the user to move to a next fillable field;

retrieving the designator associated with the selected fillable field from the data store;

determining a next designator in the sequence;

identifying, from the data store, the fillable field associated with the next designator; and

automatically causing the cursor to be displayed on the display device at the fillable field associated with the next designator.

8. The method of claim 1 , further comprising:

receiving a selection of a fillable field from a user, and causing a cursor to be displayed on a display device at the selected fillable field;

determining, by the electronic device, that the cursor should be moved to a next fillable field;

retrieving the designator associated with the selected fillable field from the data store;

determining a next designator in the sequence;

identifying, from the data store, the fillable field associated with the next designator; and

automatically causing the cursor to be displayed on the display device at the fillable field associated with the next designator.

9. The method of claim 1 , wherein receiving, by an electronic device, an electronic form comprises capturing, by an image capture device of the electronic device, the image of the document.

10. A system of identifying one or more fillable fields of an electronic form, the system comprising:

an electronic device;

a computer-readable storage medium comprising one or more programming instructions that are configured to instruct the electronic device to:

receive an electronic form, wherein the electronic form comprises an image of a document,

identify one or more fillable field candidates of the electronic form,

determine, for each fillable field candidate, whether the fillable field candidate is a fillable field by:

identifying one or more box candidates from the fillable field candidates, and

for one or more of the box candidates:

generating an out-border for the box candidate, wherein the out-border is a rectangular zone that surrounds an outer portion of the box candidate,

generating an in-border for the box candidate, wherein the in-border is a rectangular zone that encompasses at least a portion of the box candidate,

determining a histogram of pixels of the box candidate between the in-border and the out-border,

determining a ratio of black pixels to white pixels of the histogram, and

in response to the ratio exceeding a threshold value, determining that the box candidate is a fillable field, otherwise, determining that the box candidate is not a fillable field, and

update metadata associated with the electronic form by applying a sequencing framework to only the fillable fields by:

obtaining position information for each fillable field, wherein the position information indicates a position of the fillable field on the document as displayed via a display device,

sorting the fillable fields based on the position information to form a sequence of fillable fields,

determining a designator to each fillable field, wherein the designator indicates a position of a corresponding fillable field in the sequence, and

storing the designator in a data store such that it is associated with the corresponding fillable field.

11. The system of claim 10 , wherein the one or more programming instructions that are configured to instruct the electronic device to identify the one or more box candidates from the fillable field candidates comprise one or more programming instructions that are configured to instruct the electronic device to perform one or more of the following:

use a seed fill algorithm to identify the one or more box candidates from the fillable field candidates; or

use a flood fill algorithm to identify the one or more box candidates from the fillable field candidates.

12. The system of claim 10 , wherein the computer-readable storage medium further comprises one or more programming instructions that are configured to instruct the electronic device to:

in response to determining that the box candidate is a fillable field, determine whether the box candidate is a checkbox; and

in response to determining that the box candidate is a checkbox, add the box candidate to a checkbox list, otherwise, adding the box candidate to a box list.

13. The system of claim 10 , wherein the one or more programming instructions that are configured to instruct the electronic device to determine whether the fillable field candidate is a fillable field comprise one or more programming instructions that are configured to instruct the electronic device to determine whether the fillable field candidate is a line by:

obtaining dimensions for the fillable field candidate, wherein the dimensions includes a height value and a length value; and

in response to the height value not exceeding a first threshold value and the length value exceeding a second threshold value, identifying the fillable field candidate as a fillable field.

14. The system of claim 10 , wherein the one or more programming instructions that are configured to instruct the electronic device to determine whether the fillable field candidate is a fillable field comprise one or more programming instructions that are configured to instruct the electronic device to determine whether the fillable field candidate is a line by:

obtaining dimensions for the fillable field candidate, wherein the dimensions includes a height value and a length value; and

in response to the length value not exceeding a first threshold value and the height value exceeding a second threshold value, identifying the fillable field candidate as a fillable field.

15. The system of claim 10 , wherein the one or more programming instructions that are configured to instruct the electronic device to sort the fillable fields based on the position information to form a sequence of fillable fields comprise one or more programming instructions that are configured to instruct the electronic device to sort the fillable fields from a top of the document to a bottom of the document, and from a left side of the document to a right side of the document based on the position information.

16. The system of claim 10 , wherein the computer-readable storage medium further comprises one or more programming instructions that are configured to instruct the electronic device to:

receive a selection of a fillable field from a user, and causing a cursor to be displayed on a display device at the selected fillable field;

receive an indication from the user to move to a next fillable field;

retrieve the designator associated with the selected fillable field from the data store;

determine a next designator in the sequence;

identify, from the data store, the fillable field associated with the next designator; and

automatically cause the cursor to be displayed on the display device at the fillable field associated with the next designator.

17. The system of claim 10 , wherein the computer-readable storage medium further comprises one or more programming instructions that are configured to instruct the electronic device to:

receive a selection of a fillable field from a user, and causing a cursor to be displayed on a display device at the selected fillable field;

determine that the cursor should be moved to a next fillable field;

retrieve the designator associated with the selected fillable field from the data store;

determine a next designator in the sequence;

identify, from the data store, the fillable field associated with the next designator; and

automatically cause the cursor to be displayed on the display device at the fillable field associated with the next designator.

18. The system of claim 10 , wherein the one or more programming instructions that are configured to instruct the electronic device to receive an electronic form comprise one or more programming instructions that are configured to instruct the electronic device to capture, by an image capture device of the electronic device, the image of the document.

Assignments (9)

Continuity (1)