Address normalization using deep learning and address feature vectors
Generally described, one or more aspects of the present application correspond to a machine learning address normalization system. A system of deep learning networks can normalize the tokens of a free-form address into an address component hierarchy. Feature vectors representing various characters and words of the address tokens can be input into a bi-directional long short term memory network (LSTM) to generate a hidden state representation of each token, which can be individually passed through a softmax layer to generate probabilistic values of the token being each of the components in the address hierarchy. Thereafter, a conditional random field (CRF) model can select a particular address component for each token by using learned parameters to optimize a path through the collective outputs of the softmax layer for the tokens. Thus, the free-form address can be normalized to determine the values it contains for different components of a specified address hierarchy.
1. A system comprising:
a memory storing instructions for implementing a machine learning (ML) address normalization system including at least one long short term memory (LSTM) model and a conditional random fields (CRF) model; and
one or more processors in communication with the memory, the one or more processors configured by the instructions to at least:
access an address string including at least a first token and a second token;
pass the first token through the at least one LSTM model to generate a first hidden state representation of the first token;
use the first hidden state representation of the first token to generate a first set of probabilities including, for each address component of a plurality of address components in a specified hierarchy of address components, a probability that the first token belongs to the address component;
pass the second token through the at least one LSTM model to generate a second hidden state representation of the second token;
use the second hidden state representation of the second token to generate a second set of probabilities including, for each address component of the plurality of address components, a probability that the second token belongs to the address component;
analyze the first and second sets of probabilities using the CRF model to assign the first token to a first address component of the plurality of address components and to assign the second token to a second address component of the plurality of address components; and
generate normalized address data indicating that the first token is a value of the first address component and the second token is a value of the second address component.
2. The system of claim 1 , further comprising:
a word embedding model trained to generate word embeddings of input tokens;
wherein the at least one LSTM model includes a first LSTM model trained to generate character level embeddings of tokens and a second LSTM model trained to generate hidden state representations of tokens; and
wherein the one or more processors are configured by the instructions to at least:
use the first LSTM model to generate a character level embedding of the first token;
use the word embedding model to generate a word embedding of the first token; and
pass a combination of the character level embedding and the word embedding through the second LSTM model to generate the hidden state representation of the first token.
3. The system of claim 1 , wherein the specified hierarchy of address components corresponds to a particular region, wherein the ML address normalization system includes a parameter learned based at least in part on training data including a plurality of addresses from the particular region, and where the address string represents a physical location within the particular region.
4. The system of claim 1 , wherein the one or more processors are configured by the instructions to at least:
resolve the normalized address data to a geocode; and
generate a delivery route plan based at least in part on the geocode.
5. A computer-implemented method comprising:
accessing an address string including at least a first token and a second token;
passing the first token through at least one sequence analysis model to generate a first hidden state representation of the first token;
using the first hidden state representation of the first token to generate a first set of probabilities including, for each address component of a plurality of address components in a specified hierarchy of address components, a probability that the first token belongs to the address component;
passing the second token through the at least one sequence analysis model to generate a second hidden state representation of the second token;
using the second hidden state representation of the second token to generate a second set of probabilities including, for each address component of the plurality of address components, a probability that the second token belongs to the address component;
based at least in part on the first and second sets of probabilities, assigning the first token to a first address component of the plurality of address components and assigning the second token to a second address component of the plurality of address components; and
generating normalized address data indicating that the first token is a value of the first address component and the second token is a value of the second address component.
6. The computer-implemented method of claim 5 , further comprising:
optimizing a path through the first and second sets of probabilities using a conditional random field (CRF) model;
assigning the first token to the first address component based at least in part on a location of the path through the first set of probabilities; and
assigning the second token to the second address component based at least in part on a location of the path through the first set of probabilities.
7. The computer-implemented method of claim 6 , wherein the specified hierarchy of address components corresponds to a particular region, the computer-implemented method further comprising training parameters of the at least one sequence analysis model and the CRF model based at least in part on training data including a plurality of addresses from the particular region.
8. The computer-implemented method of claim 5 , wherein the first address component and the second address component are the same address component in the specified hierarchy of address components, the computer-implemented method further comprising:
combining the first and second tokens into a string; and
setting a value of the same address component to the string to generate the normalized address data.
9. The computer-implemented method of claim 5 , further comprising determining the first address component and the second address component as different address components in the specified hierarchy of address components.
10. The computer-implemented method of claim 5 , wherein the at least one sequence analysis model includes a first LSTM model trained to generate character level embeddings of tokens and a second LSTM model trained to generate hidden state representations of tokens, the computer-implemented method further comprising:
using a word embedding model trained to generate a word embedding of the first token;
using the first LSTM model to generate a character level embedding of the first token; and
passing a combination of the character level embedding and the word embedding through the second LSTM model to generate the hidden state representation of the first token.
11. The computer-implemented method of claim 10 , wherein the first LSTM model is a bi-directional LSTM, the computer-implemented method further comprising generating the character level embedding as a combination of a first vector generated by analyzing characters of the first token in a forward order and a second vector generated by analyzing the characters of the first token in a backward order.
12. The computer-implemented method of claim 5 , further comprising:
resolving the normalized address data to a geocode; and
generating a delivery route plan based at least in part on the geocode.
13. The computer-implemented method of claim 5 , further comprising isolating the first token and the second token by identifying a specified punctuation between the first token and the second token in the address string.
14. A non-transitory computer-readable medium comprising computer-executable instructions that, when executed on a computing system, cause the computing system to perform operations comprising:
accessing an address string including at least a first token and a second token;
passing the first token through at least one sequence analysis model to generate a first output representing the first token;
using the first output to generate a first set of probabilities including, for each address component of a plurality of address components in a specified hierarchy of address components, a probability that the first token belongs to the address component;
passing the second token through the at least one sequence analysis model to generate a second output representing the second token;
using the second output to generate a second set of probabilities including, for each address component of the plurality of address components, a probability that the second token belongs to the address component;
based at least in part on the first and second sets of probabilities, assigning the first token to a first address component of the plurality of address components and assigning the second token to a second address component of the plurality of address components; and
generating normalized address data indicating that the first token is a value of the first address component and the second token is a value of the second address component.
15. The non-transitory computer-readable medium of claim 14 , the operations further comprising:
optimizing a path through the first and second sets of probabilities using a conditional random fields (CRF) model;
assigning the first token to the first address component based at least in part on a location of the path through the first set of probabilities; and
assigning the second token to the second address component based at least in part on a location of the path through the first set of probabilities.
16. The non-transitory computer-readable medium of claim 15 , wherein the specified hierarchy of address components corresponds to a particular region, the operations further comprising training parameters of the at least one sequence analysis model and the CRF model based at least in part on training data including a plurality of addresses from the particular region.
17. The non-transitory computer-readable medium of claim 14 , wherein the first address component and the second address component are the same address component in the specified hierarchy of address components, the operations further comprising:
combining the first and second tokens into a string; and
setting a value of the same address component to the string to generate the normalized address data.
18. The non-transitory computer-readable medium of claim 14 , wherein the at least one sequence analysis model includes a first LSTM model trained to generate character level embeddings of tokens and a second LSTM model trained to generate hidden state representations of tokens, the operations further comprising:
using a word embedding model trained to generate a word embedding of the first token;
using the first LSTM model to generate a character level embedding of the first token; and
passing a combination of the character level embedding and the word embedding through the second LSTM model to generate the hidden state representation of the first token.
19. The non-transitory computer-readable medium of claim 14 , the operations further comprising:
resolving the normalized address data to a geocode; and
generating a delivery route plan based at least in part on the geocode.
20. The non-transitory computer-readable medium of claim 14 , the operations further comprising isolating the first token and the second token by identifying a specified punctuation between the first token and the second token in the address string.