Multiple channels of rasterized content for page decomposition using machine learning
Techniques are provided for identifying structural elements of a document. One Methodology includes generating a first channel of rasterized content by rasterizing a full page of the document and generating one or more additional channels of rasterized content from the page of the document by rasterizing one or more corresponding content types from the page of the document. Each of the one or more additional channels includes a specific type of content that is different from each of the other one or more additional channels. The methodology further includes inputting the first channel of rasterized content and the one or more additional channels of rasterized content into a machine learning (ML) model. The methodology continues with determining location and classification for each of a plurality of structural elements on the page of the document using the ML model.
1. A method for identifying structural elements of a document, the method comprising:
generating a first channel of rasterized content by rasterizing a full page of the document;
generating one or more additional channels of rasterized content from the page of the document by rasterizing one or more corresponding content types from the page of the document, wherein each of the one or more additional channels includes a specific type of content that is different from each of the other one or more additional channels;
inputting the first channel of rasterized content and the one or more additional channels of rasterized content into a machine learning (ML) model; and
determining location and classification for each of a plurality of structural elements on the page of the document using the ML model based at least on the first channel of rasterized content and the one or more additional channels of rasterized content.
2. The method of claim 1 , wherein rasterizing one or more corresponding content types comprises rasterizing text structures from the page of the document.
3. The method of claim 1 , wherein rasterizing one or more corresponding content types comprises rasterizing graphic structures from the page of the document.
4. The method of claim 1 , wherein rasterizing one or more corresponding content types comprises rasterizing image structures from the page of the document.
5. The method of claim 1 , wherein the one or more additional channels of rasterized content include second, third and fourth channels, and rasterizing the one or more corresponding content types comprises at least two of: rasterizing text structures from the page of the document to provide the second channel of rasterized content, rasterizing graphic structures from the page of the document to provide the third channel of rasterized content, and rasterizing image structures from the page of the document to provide the fourth channel of rasterized content.
6. The method of claim 1 , further comprising generating one or more other additional channels of rasterized content by: rasterizing one or more full pages sequentially after the page of the document, and/or rasterizing one or more full pages sequentially before the page of the document, and wherein the inputting further comprises inputting the one or more other additional channels into the ML model.
7. The method of claim 6 , wherein the inputting comprises creating a stack of layers for input to the ML model, wherein a first layer of the stack includes data from the first channel of rasterized content and one or more additional layers of the stack include data from the one or more additional channels of rasterized content and data from the one or more other additional channels of rasterized content, respectively.
8. The method of claim 1 , further comprising generating additional other channels of rasterized content, wherein each of the additional other channels is associated with a single color, and wherein the inputting further comprises inputting the additional other channels of rasterized content into the ML model.
9. A system configured to identify structural elements of a document, the system comprising:
at least one processor; and
a storage medium operatively coupled to the at least one processor and configured to store instructions that when executed by the at least one processor cause the at least one processor to perform operations comprising
generating a first channel of rasterized content by rasterizing a full page of the document,
generating at least one of
a second channel of rasterized content from the page of the document by rasterizing text structures from the page of the document,
a third channel of rasterized content from the page of the document by rasterizing graphic structures from the page of the document, or
a fourth channel of rasterized content from the page of the document by rasterizing image structures from the page of the document,
inputting the first channel of rasterized content and one or more of the second, third and fourth channels of rasterized content into a machine learning (ML) model, and
determining location and classification for each of a plurality of structural elements on the page of the document using the ML model based at least on the first channel of rasterized content and the one or more additional channels of rasterized content.
10. The system of claim 9 , wherein the inputting includes inputting two or more of the second, third and fourth channels of rasterized content into the ML model.
11. The system of claim 9 , wherein the operations further comprise generating one or more additional channels of rasterized content by: rasterizing one or more full pages sequentially after the page of the document, and/or rasterizing one or more full pages sequentially before the page of the document, and wherein the inputting further comprises inputting the one or more other additional channels into the ML model.
12. The system of claim 11 , wherein the inputting comprises creating a stack of layers for input to the ML model, wherein a first layer of the stack includes data from the first channel of rasterized content and one or more additional layers of the stack include data from the second, third, fourth, and the one or more additional channels of rasterized content, respectively.
13. A computer program product including one or more non-transitory machine-readable media having instructions encoded thereon that when executed by at least one processor causes a process for identifying structural elements of a document to be carried out, the process comprising:
generating a first channel of rasterized content by rasterizing a full page of the document;
generating one or more additional channels of rasterized content from the page of the document by rasterizing one or more corresponding content types from the page of the document, wherein each of the one or more additional channels includes a specific type of content that is different from each of the other one or more additional channels;
inputting the first channel of rasterized content and the one or more additional channels of rasterized content into a machine learning (ML) model; and
determining location and classification for each of a plurality of structural elements on the page of the document using the ML model based at least on the first channel of rasterized content and the one or more additional channels of rasterized content.
14. The computer program product of claim 13 , wherein rasterizing one or more corresponding content types comprises rasterizing text structures from the page of the document.
15. The computer program product of claim 13 , wherein rasterizing one or more corresponding content types comprises rasterizing graphic structures from the page of the document.
16. The computer program product of claim 13 , wherein rasterizing one or more corresponding content types comprises rasterizing image structures from the page of the document.
17. The computer program product of claim 13 , wherein the one or more additional channels of rasterized content include second, third and fourth channels, and rasterizing the one or more corresponding content types comprises at least two of: rasterizing text structures from the page of the document to provide the second channel of rasterized content, rasterizing graphic structures from the page of the document to provide the third channel of rasterized content, and rasterizing image structures from the page of the document to provide the fourth channel of rasterized content.
18. The computer program product of claim 13 , further comprising generating one or more other additional channels of rasterized content by: rasterizing one or more full pages sequentially after the page of the document, and/or rasterizing one or more full pages sequentially before the page of the document, and wherein the inputting further comprises inputting the one or more other additional channels into the ML model.
19. The computer program product of claim 18 , wherein the inputting comprises creating a stack of layers for input to the ML model, wherein a first layer of the stack includes data from the first channel of rasterized content and one or more additional layers of the stack include data from the one or more additional channels of rasterized content and data from the one or more other additional channels of rasterized content, respectively.
20. The computer program product of claim 13 , further comprising generating additional other channels of rasterized content, wherein each of the additional other channels is associated with a single color, and wherein the inputting further comprises inputting the additional other channels of rasterized content into the ML model.