Extraction of genealogy data from obituaries
Systems, methods, and other techniques for extracting data from obituaries are provided. In some embodiments, an obituary containing a plurality of words is received. Using a machine learning model, an entity tag from a set of entity tags may be assigned to each of one or more words of the plurality of words. Each particular tag from the set of entity tags may include a relationship component and a category component. The relationship component may indicate a relationship between a particular word and the deceased individual. The category component may indicate a categorization of the particular word to a particular category from a set of categories. The extracted data may be stored in a genealogical database.
1. A computer-implemented method for extracting data from obituaries, the method comprising:
receiving an image;
recognizing text in the image;
determining that the image contains at least one obituary;
segmenting the image into a plurality of sections;
determining that a section of the plurality of sections contains an obituary of the at least one obituary, the obituary containing a plurality of words and corresponding to a deceased individual; and
assigning, using an entity tagging machine learning (ML) model, an entity tag from a set of entity tags to each of one or more words of the plurality of words, wherein each particular entity tag from the set of entity tags includes a relationship component and a category component, wherein the relationship component indicates a relationship between a particular word of the plurality of words to which the particular entity tag is assigned and the deceased individual, and wherein the category component indicates a categorization of the particular word to a particular category from a set of categories;
wherein, prior to assigning the entity tag, the entity tagging ML model is trained by:
receiving a plurality of input words corresponding to an input obituary;
creating a first training set based on the plurality of input words;
training the entity tagging ML model in a first stage using the first training set;
creating a second training set including a subset of the plurality of input words to which entity tags were incorrectly assigned after the first stage; and
training the entity tagging ML model in a second stage using the second training set.
2. The computer-implemented method of claim 1 , wherein the entity tagging ML model is a neural network.
3. The computer-implemented method of claim 1 , wherein the relationship component is selected from the group comprising: SELF, SPOUSE, CHILD, SIBLING, and PARENT.
4. The computer-implemented method of claim 1 , wherein the category component is selected from the group comprising: PERSON, PLACE, DATE, and AGE.
5. The computer-implemented method of claim 1 , further comprising:
predicting, using a gender prediction ML model, a gender for each of the plurality of words for which the category component of the particular entity tag that is assigned is PERSON.
6. The computer-implemented method of claim 1 , further comprising:
assigning, using a name assignment ML model, a name part tag from a set of name part tags to each of the plurality of words for which the category component of the particular entity tag that is assigned is PERSON, wherein the set of name part tags at least includes GIVEN NAME, SURNAME, and MAIDEN NAME.
7. A computer-implemented method for extracting data from obituaries, the method comprising:
receiving an image;
segmenting the image into a plurality of sections;
determining that a section of the plurality of sections contains an obituary containing a plurality of words, the obituary corresponding to a deceased individual; and
assigning, using an entity tagging machine learning (ML) model, an entity tag from a set of entity tags to each of one or more words of the plurality of words, wherein each particular entity tag from the set of entity tags includes a relationship component and a category component, wherein the relationship component indicates a relationship between a particular word of the plurality of words to which the particular entity tag is assigned and the deceased individual, and wherein the category component indicates a categorization of the particular word to a particular category from a set of categories;
wherein, prior to assigning the entity tag, the entity tagging ML model is trained by:
receiving an input obituary containing a plurality of input words;
creating a first training set based on the plurality of input words;
training the entity tagging ML model in a first stage using the first training set;
creating a second training set including a subset of the plurality of input words to which entity tags were incorrectly assigned after the first stage; and
training the entity tagging ML model in a second stage using the second training set.
8. The computer-implemented method of claim 7 , wherein the entity tagging ML model is a neural network.
9. The computer-implemented method of claim 7 , wherein the relationship component is selected from the group comprising: SELF, SPOUSE, CHILD, SIBLING, and PARENT.
10. The computer-implemented method of claim 7 , wherein the category component is selected from the group comprising: PERSON, PLACE, DATE, and AGE.
11. The computer-implemented method of claim 7 , further comprising:
recognizing text in the image.
12. The computer-implemented method of claim 7 , further comprising:
predicting, using a gender prediction ML model, a gender for each of the plurality of words for which the category component of the particular entity tag that is assigned is PERSON.
13. The computer-implemented method of claim 7 , further comprising:
assigning, using a name assignment ML model, a name part tag from a set of name part tags to each of the plurality of words for which the category component of the particular entity tag that is assigned is PERSON, wherein the set of name part tags at least includes GIVEN NAME and SURNAME.
14. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving an obituary containing a plurality of words, the obituary corresponding to a deceased individual; and
assigning, using an entity tagging machine learning (ML) model, an entity tag from a set of entity tags to each of one or more words of the plurality of words, wherein each particular entity tag from the set of entity tags includes a relationship component and a category component, wherein the relationship component indicates a relationship between a particular word of the plurality of words to which the particular entity tag is assigned and the deceased individual, and wherein the category component indicates a categorization of the particular word to a particular category from a set of categories;
wherein, prior to assigning the entity tag, the entity tagging ML model is trained by:
receiving an input obituary containing a plurality of input words;
creating a first training set based on the plurality of input words;
training the entity tagging ML model in a first stage using the first training set;
creating a second training set including a subset of the plurality of input words to which entity tags were incorrectly assigned after the first stage; and
training the entity tagging ML model in a second stage using the second training set.
15. The non-transitory computer-readable medium of claim 14 , wherein the entity tagging ML model is a neural network.
16. The non-transitory computer-readable medium of claim 14 , wherein the relationship component is selected from the group comprising: SELF, SPOUSE, CHILD, SIBLING, and PARENT.
17. The non-transitory computer-readable medium of claim 14 , wherein the category component is selected from the group comprising: PERSON, PLACE, DATE, and AGE.