Topic segmentation of image-derived text
Described herein are systems, methods, and other techniques for segmenting an input text. A set of tokens are extracted from the input text. Token representations are computed for the set of tokens. The token representations are provided to a machine learning model that generates a set of label predictions corresponding to the set of tokens. The machine learning model was previously trained to generate label predictions in response to being provided input token representations. Each of the set of label predictions indicates a position of a particular token of the set of tokens with respect to a particular segment. One or more segments within the input text are determined based on the set of label predictions.
1. A computer-implemented method of segmenting an input text, the method comprising:
extracting a set of tokens from the input text, wherein the set of tokens comprising a first token that represents a natural person, a second token that represents a relationship associated with the natural person;
computing token representations for the set of tokens, the token representations comprising a plurality of embedding vectors;
providing one or more of the embedding vectors to a machine learning model that generates a set of label predictions corresponding to the set of tokens, wherein the machine learning model was previously trained to generate label predictions in response to being provided input embedding vectors, and wherein each of the set of label predictions indicates a position of a particular token of the set of tokens with respect to a particular segment; and
determining one or more segments within the input text based on the set of label predictions.
2. The computer-implemented method of claim 1 , further comprising:
receiving an image; and
generating the input text based on the image using a character recognizer.
3. The computer-implemented method of claim 2 , wherein computing the token representations for the set of tokens includes:
computing a position vector for each of the set of tokens, wherein the position vector indicates a location of a token with respect to a physical reference point within the image.
4. The computer-implemented method of claim 2 , wherein the character recognizer is an optical character reader.
5. The computer-implemented method of claim 4 , wherein the position of the particular token with respect to the particular segment is one of:
at a beginning of the particular segment;
inside the particular segment; or
outside the particular segment.
6. The computer-implemented method of claim 5 , wherein the image includes a plurality of marriage announcements captured from a newspaper.
7. The computer-implemented method of claim 1 , wherein the machine learning model includes a bi-directional long short-term memory (LSTM) layer.
8. The computer-implemented method of claim 1 , wherein computing the token representations for the set of tokens includes at least one of:
computing an ELMo embedding for each of the set of tokens using a trained ELMo model; or
computing a GloVe embedding for each of the set of tokens using a trained GloVe model.
9. A computer-readable hardware storage device comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations for segmenting an input text, the operations comprising:
extracting a set of tokens from the input text, wherein the set of tokens comprising a first token that represents a natural person, a second token that represents a relationship associated with the natural person;
computing token representations for the set of tokens, the token representations comprising a plurality of embedding vectors;
providing one or more of the embedding vectors to a machine learning model that generates a set of label predictions corresponding to the set of tokens, wherein the machine learning model was previously trained to generate label predictions in response to being provided input embedding vectors, and wherein each of the set of label predictions indicates a position of a particular token of the set of tokens with respect to a particular segment; and
determining one or more segments within the input text based on the set of label predictions.
10. The computer-readable hardware storage device of claim 9 , wherein the operations further comprise:
receiving an image; and
generating the input text based on the image using a character recognizer.
11. The computer-readable hardware storage device of claim 10 , wherein computing the token representations for the set of tokens includes:
computing a position vector for each of the set of tokens, wherein the position vector indicates a location of a token with respect to a physical reference point within the image.
12. The computer-readable hardware storage device of claim 10 , wherein the character recognizer is an optical character reader.
13. The computer-readable hardware storage device of claim 12 , wherein the position of the particular token with respect to the particular segment is one of:
at a beginning of the particular segment;
inside the particular segment; or
outside the particular segment.
14. The computer-readable hardware storage device of claim 13 , wherein the image includes a plurality of marriage announcements captured from a newspaper.
15. The computer-readable hardware storage device of claim 9 , wherein the machine learning model includes a bi-directional long short-term memory (LSTM) layer.
16. The computer-readable hardware storage device of claim 9 , wherein computing the token representations for the set of tokens includes at least one of:
computing an ELMo embedding for each of the set of tokens using a trained ELMo model; or
computing a GloVe embedding for each of the set of tokens using a trained GloVe model.
17. A system for segmenting an input text, the system comprising:
one or more processors; and
a computer-readable medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
extracting a set of tokens from the input text, wherein the set of tokens comprising a first token that represents a natural person, a second token that represents a relationship associated with the natural person;
computing token representations for the set of tokens, the token representations comprising a plurality of embedding vectors;
providing one or more of the embedding vectors to a machine learning model that generates a set of label predictions corresponding to the set of tokens, wherein the machine learning model was previously trained to generate label predictions in response to being provided input embedding vectors, and wherein each of the set of label predictions indicates a position of a particular token of the set of tokens with respect to a particular segment; and
determining one or more segments within the input text based on the set of label predictions.
18. The system of claim 17 , wherein the operations further comprise:
receiving an image; and
generating the input text based on the image using a character recognizer.
19. The system of claim 18 , wherein computing the token representations for the set of tokens includes:
computing a position vector for each of the set of tokens, wherein the position vector indicates a location of a token with respect to a physical reference point within the image.
20. The system of claim 18 , wherein the position of the particular token with respect to the particular segment is one of:
at a beginning of the particular segment;
inside the particular segment; or
outside the particular segment.