Text Extraction Heuristics
Systems and methods are described for facilitating reliable extraction of text content from particular classes of digital documents (e.g., PDF documents) having that lack the structural information traditionally necessary for straightforward text extraction. Font encoding patterns are identified as indicative of an offset that may be applied to code values in the font encoding to produce values of intended Unicode characters. By classifying a particular digital document according to these patterns, an offset may be identified and text may be extracted without requiring use of computationally intensive optical character recognition (OCR) techniques.
1 . A computer-implemented method for facilitating text content extraction from a digital document, the method comprising:
identifying, via one or more processors, a font object corresponding to a content stream of a digital document, the font object comprising a font encoding of a plurality of glyph codes to a respective plurality of glyphs;
determining, via the one or more processors, one or more characteristics of one or more glyphs of the font object, the one or more characteristics excluding text information content of the one or more glyphs; and
determining, via the one or more processors based upon the one or more determined glyph characteristics, an integer offset associated with the font encoding,
wherein, for each particular glyph code of the plurality of glyph codes in the font encoding, adding the integer offset to the particular glyph code produces a respective sum value corresponding to a respective Unicode character, the Unicode character being a character represented by the glyph encoded at the particular glyph code of the font encoding.
2 . The computer-implemented method of claim 1 , wherein the digital document is Portable Document Format (PDF) document.
3 . The computer-implemented method of claim 1 , further comprising adding, via the one or more processors, the integer offset to each glyph code in the font encoding to produce the respective sum values corresponding to the respective Unicode characters.
4 . The computer-implemented method of claim 3 , further comprising:
creating, via the one or more processors, a mapping object comprising, for each particular glyph code of the plurality of glyph codes of the font encoding, an indicator of the respective sum value corresponding to the particular glyph code; and
modifying, via the one or more processors, the digital document to include the created mapping object.
5 . The computer-implemented method of claim 3 , further comprising:
modifying, via the one or more processors, the font object of the digital document, wherein modifying the font object includes, for each particular glyph code of the plurality of glyph codes, replacing the glyph code with the respective sum value corresponding to the respective Unicode character; and
modifying, via the one or more processors, the content stream of the digital document, wherein modifying the content stream includes, for each instance of a glyph code in the content stream, replacing the content stream glyph code with the respective sum value corresponding to the respective Unicode character.
6 . The computer-implemented method of claim 1 , wherein determining the integer offset includes determining the offset having a hexadecimal value of 0x1D.
7 . The computer-implemented method of claim 1 , wherein determining the integer offset includes determining the offset having a hexadecimal value of 0x1E.
8 . The computer-implemented method of claim 1 , further comprising:
for each of one or more glyph codes in the content stream of the digital document, adding, via the one or more processors, the offset to the content stream glyph code to produce a respective content stream sum value; and
determining, via the one or more processors, based upon the one or more respective content stream sum values, one or more corresponding Unicode characters to thereby extract the text content of the content stream.
9 . The computer-implemented method of claim 3 , further comprising:
for each of one or more glyph codes in the content stream of the digital document, referencing, via the one or more processors, the created mapping object to identify a respective mapped value corresponding to the content stream glyph code; and
determining, via the one or more processors, based upon the one or more respective content stream mapped values, one or more corresponding Unicode characters to thereby extract the text content of the content stream.
10 . The computer-implemented method of claim 1 , wherein determining the one or more the one or more glyph characteristics comprises determining a width of one or more glyphs of the font object.
11 . The computer-implemented method of claim 1 , wherein determining the one or more glyph characteristics comprises determining whether a glyph of the font object is set, based at least in part upon at least one of (1) a comparison of a width of the glyph to a default width of the font object, or (2) a comparison of the width of the glyph to a predicted space width of the font object.
12 . A computing system configured to facilitate text content extraction from a digital document, the computing system comprising:
one or more processors; and
one or more non-transitory computer memories storing computer-executable instructions that, when executed via the one or more processors, cause the one or more processors to:
identify a font object corresponding to a content stream of a digital document, the font object comprising a font encoding of a plurality of glyph codes to a respective plurality of glyphs,
determine one or more characteristics of one or more glyphs of the font object, the one or more characteristics excluding text information content of the one or more glyphs, and
determine, based upon the one or more determined characteristics of the one or more glyphs, an integer offset associated with the font encoding,
wherein, for each particular glyph code of the plurality of glyph codes in the font encoding, adding the integer offset to the particular glyph code produces a respective sum value corresponding to a respective Unicode character, the Unicode character being a character represented by the glyph encoded at the particular glyph code of the font encoding.
13 . The computing system of claim 12 , wherein the digital document is a PDF document.
14 . The computing system of claim 12 , wherein the computer-executable instructions further include instructions that, when executed via the one or more processors, cause the one or more processors to:
add the integer offset to each glyph code in the font encoding to produce the respective sum values corresponding to the respective Unicode characters.
15 . The computing system of claim 14 , wherein the computer-executable instructions further include instructions that, when executed via the one or more processors, cause the one or more processors to:
create a mapping object comprising, for each particular glyph code of the plurality of glyph codes of the font encoding, an indicator of the respective sum value corresponding to the particular glyph code; and
modify the digital document to include the created mapping object.
16 . The computing system of claim 14 , wherein the computer-executable instructions further include instructions that, when executed via the one or more processors, cause the one or more processors to:
modify the font object of the digital document, wherein modifying the font object includes, for each particular glyph code of the plurality of glyph codes, replacing the glyph code with the respective sum value corresponding to the respective Unicode character; and
modify the content stream of the digital document, wherein modifying the content stream includes, for each instance of a glyph code in the content stream, replacing the content stream glyph code with the respective sum value corresponding to the respective Unicode character.
17 . The computing system of claim 12 , wherein the determined integer offset has a hexadecimal value of 0x1D.
18 . The computing system of claim 12 , wherein the determined integer offset has a hexadecimal value of 0x1E.
19 . The computing system of claim 12 , wherein the computer-executable instructions further include instructions that, when executed via the one or more processors, cause the one or more processors to:
for each of one or more glyph codes in the content stream of the digital document, add the offset to the content stream glyph code to produce a respective content stream sum value; and
determining, based upon the one or more respective content stream sum values, one or more corresponding Unicode characters to thereby extract the text content of the content stream.
20 . One or more non-transitory computer-readable media storing computer-executable instructions that, when executed via a computer, cause the computer to:
identify a font object corresponding to a content stream of a digital document, the font object comprising a font encoding of a plurality of glyph codes to a respective plurality of glyphs,
determine one or more characteristics of one or more glyphs of the font object, the one or more characteristics excluding text information content of the one or more glyphs, and
determine, based upon the one or more determined characteristics of the one or more glyphs, an integer offset associated with the font encoding,
wherein, for each particular glyph code of the plurality of glyph codes in the font encoding, adding the integer offset to the particular glyph code produces a respective sum value corresponding to a respective Unicode character, the Unicode character being a character represented by the glyph encoded at the particular glyph code of the font encoding.