Simulation of label data to optimize the visual document understanding by using PDFs annotation aware methodology
One example method includes obtaining a text-based document, identifying annotations in the text-based document, and retaining those annotations, converting the text-based document to an image, processing the image, creating simulated label data by integrating the processed image with the annotations, and using the processed simulated label data to train a machine learning model of an OCR (optical character recognition) system. The processed image is a lower quality version of the text-based document that was used to create the image.
1 . A method, comprising:
obtaining a text-based document;
identifying annotations in the text-based document, and retaining those annotations as native document annotations that exist in the text-based document prior to conversion of the text-based document to an image;
converting the text-based document to an image;
processing the image using one or more image degradation operations selected to simulate low-quality scanned documents, the processing including at least one of document smoothening, noise introduction, random patch smoothening, or edge detection and removal;
creating simulated label data by integrating the processed image with the annotations as training labels by transforming the retained annotations to correspond to the processed image while preserving spatial and semantic alignment between the annotations and document content despite the image degradation; and
using the processed simulated label data to train a machine learning model of an OCR (optical character recognition) system using the annotations as ground-truth labels for the degraded image.
2 . The method as recited in claim 1 , wherein the text-based document is a text-based PDF (portable document format) document.
3 . The method as recited in claim 1 , wherein the annotations were generated by a human.
4 . The method as recited in claim 1 , wherein the annotations comprise text and/or one or more bounding boxes.
5 . The method as recited in claim 1 , wherein processing the image comprises performing a document smoothening process on the image.
6 . The method as recited in claim 1 , wherein processing the image comprises performing a noise introduction process on the image.
7 . The method as recited in claim 1 , wherein processing the image comprises performing a random patch smoothening process on the image.
8 . The method as recited in claim 1 , wherein processing the image comprises performing an edge detection and removal process on the image.
9 . The method as recited in claim 1 , wherein a verification operation is performed that confirms that the annotations that were retained match the annotations in the image.
10 . The method as recited in claim 1 , wherein the processed image is a lower quality version of the text-based document.
11 . A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
obtaining a text-based document;
identifying annotations in the text-based document, and retaining those annotations as native document annotations that exist in the text-based document prior to conversion of the text-based document to an image;
converting the text-based document to an image;
processing the image using one or more image degradation operations selected to simulate low-quality scanned documents, the processing including at least one of document smoothening, noise introduction, random patch smoothening, or edge detection and removal;
creating simulated label data by integrating the processed image with the annotations as training labels by transforming the retained annotations to correspond to the processed image while preserving spatial and semantic alignment between the annotations and document content despite the image degradation; and
using the processed simulated label data to train a machine learning model of an OCR (optical character recognition) system using the annotations as ground-truth labels for the degraded image.
12 . The non-transitory storage medium as recited in claim 11 , wherein the text-based document is a text-based PDF (portable document format) document.
13 . The non-transitory storage medium as recited in claim 11 , wherein the annotations were generated by a human.
14 . The non-transitory storage medium as recited in claim 11 , wherein the annotations comprise text and/or one or more bounding boxes.
15 . The non-transitory storage medium as recited in claim 11 , wherein processing the image comprises performing a document smoothening process on the image.
16 . The non-transitory storage medium as recited in claim 11 , wherein processing the image comprises performing a noise introduction process on the image.
17 . The non-transitory storage medium as recited in claim 11 , wherein processing the image comprises performing a random patch smoothening process on the image.
18 . The non-transitory storage medium as recited in claim 11 , wherein processing the image comprises performing an edge detection and removal process on the image.
19 . The non-transitory storage medium as recited in claim 11 , wherein a verification operation is performed that confirms that the annotations that were retained match the annotations in the image.
20 . The non-transitory storage medium as recited in claim 11 , wherein the processed image is a lower quality version of the text-based document.