Granted Patent B2

US 12,314,657 · App. 17/501,417 · Granted May 27, 2025

Inventor: Manbinder Pal Singh (Coral Springs, FL)

G06F40/174G06V30/262G06V30/412G06V30/414G06F3/04842

Abstract

Claims (55)

1. A method comprising:

locating, by a computing device, a portion of a digital image that depicts fields of a paper form, the fields including markings;

identifying, by the computing device, a marking within one field of the fields of the paper form as being contrary to a set of rules;

performing, by the computing device, computer-vision analysis of a geometry of pixels of the digital image to locate one or more geometric shapes associated with one or more field prototypes;

locating, based on the computer-vision analysis, a first geometric shape in the one field, wherein the first geometric shape is associated with the marking within the one field;

locating, based on the computer-vision analysis. a second geometric shape without any marking, wherein the second geometric shape is similar to the first geometric shape;

modifying, by the computing device, the digital image to remove the identified marking from the digital image by replacing the first geometric shape with the second geometric shape; and

causing to display, by the computing device and on a user device, of the portion of the digital image depicting the one field of the paper form as being unmarked.

2. The method of claim 1 , wherein:

the set of rules comprises contextual consistency rules; and

identifying the marking within the one field as being contrary to a set of rules comprises:

determining contexts for the fields;

detecting contextual errors in the markings within the fields with respect to the contextual consistency rules; and

selecting one of the detected contextual errors as the identified marking.

3. The method of claim 2 , wherein determining the contexts for the fields comprises:

locating descriptive text that fits a placement pattern with respect to one or more of the fields;

determining meaning of the located descriptive text; and

identifying a context for the one or more fields based on applying a set of contextual meaning rules to the determined text.

4. The method of claim 3 , wherein locating the descriptive text that fits the placement pattern with respect to the one or more located fields comprises locating descriptive text that is within a predefined distance of the one or more located fields in one of a predefined set of orientations.

5. The method of claim 3 , wherein:

the located descriptive text comprises first text located in proximity to a first field of the one or more of the fields and second text located in proximity to a second field of the one or more of the fields;

determining the meaning of the located descriptive text comprises identifying the first text as being affirmative and the second text as being negative;

identifying the context for the one or more fields based on applying the set of contextual meaning rules to the determined text comprises determining that the first field and the second field are mutually-exclusive alternatives; and

detecting contextual errors in the markings within the fields with respect to the contextual consistency rules comprises determining that both the first field and the second field including markings is a contextual error.

6. The method of claim 2 , wherein detecting the contextual errors in the markings within the fields comprises detecting crossed-out regions using cutout analysis.

7. The method of claim 2 , wherein selecting the one of the detected contextual errors as the identified marking comprises:

presenting the detected contextual errors to a user via a graphical user interface (GUI); and

receiving a selection of the one of the detected contextual errors via the GUI.

8. The method of claim 7 , wherein identifying the marking further comprises updating the contextual consistency rules using machine learning based on the selection received from the user via the GUI.

9. The method of claim 1 , wherein modifying the digital image to remove the identified marking from the digital image comprises replacing the second geometric shape with a corresponding first geometric shape located elsewhere in the digital image.

10. The method of claim 1 , wherein locating the geometric shapes associated with one or more field prototypes comprises locating checkboxes having a square shape with a side length within a given size range.

11. The method of claim 1 , wherein locating the geometric shapes associated with one or more field prototypes comprises locating text entry fields having ruled lines parallel to a direction of text flow.

12. The method of claim 1 , wherein the method further comprises capturing the digital image from the paper form using image acquisition hardware.

13. The method of claim 1 , wherein modifying the digital image further comprises removing all identified markings from the digital image, so that the portion of the digital image depicts all the fields of the paper form as being unmarked.

14. The method of claim 13 , wherein modifying the digital image further comprises replacing the removed markings with markings of a different color than the removed markings.

15. The method of claim 1 , wherein modifying the digital image to remove the identified marking from the digital image comprises erasing pixels of the identified marking and filling in the erased pixels with a background color of the digital image except where the one field of the paper form has a different color.

16. A computer program product comprising a non-transitory computer-readable storage medium storing instructions, which, when performed by a computing device, cause the computing device to:

locate a portion of a digital image that depicts fields of a paper form, the fields including markings;

identify a marking within one field of the fields of the paper form as being contrary to a set of rules;

perform computer-vision analysis of a geometry of pixels of the digital image to locate one or more geometric shapes associated with one or more field prototypes;

locate, based on the computer-vision analysis. a first geometric shape in the one field, wherein the first geometric shape is associated with the marking within the one field;

locating, based on the computer-vision analysis, a second geometric shape without any marking. wherein the second geometric shape is similar to the first geometric shape;

modify the digital image to remove the identified marking from the digital image by replacing the first geometric shape with the second geometric shape; and

cause to display, on a user device, of the portion of the digital image depicting the one field of the paper form as being unmarked.

17. The computer program product of claim 16 , wherein the instructions, when performed by the computing device, further cause the computing device to locate the geometric shapes associated with the one or more field prototypes by locating checkboxes having a square shape with a side length within a given size range.

18. The computer program product of claim 16 , wherein the instructions, when performed by the computing device, further cause the computing device to locate the geometric shapes associated with the one or more field prototypes by locating text entry fields having ruled lines parallel to a direction of text flow.

19. An apparatus comprising processing circuitry coupled to memory configured to:

locate a portion of a digital image that depicts fields of a paper form, the fields including markings;

identify a marking within one field of the fields of the paper form as being contrary to a set of rules;

perform computer-vision analysis of a geometry of pixels of the digital image to locate one or more geometric shapes associated with one or more field prototypes;

locate, based on the computer-vision analysis, a first geometric shape in the one field, wherein the first geometric shape is associated with the marking within the one field;

locate, based on the computer-vision analysis, a second geometric shape without any marking, wherein the second geometric shape is similar to the first geometric shape;

modify the digital image to remove the identified marking from the digital image by replacing the first geometric shape with the second geometric shape; and

cause to display, on a user device, of the portion of the digital image depicting the one field of the paper form as being unmarked.

20. The apparatus of claim 19 , wherein the apparatus further comprises image acquisition hardware configured to capture the digital image from the paper form.

Assignments (9)

Continuity (1)

References Cited (8)