Granted Patent B2

US 12,361,213 · App. 17/658,416 · Granted Jul 15, 2025

Inventors: Zhongkun Jin (Ridgefield, CT); Louis Ben Arous (Philadelphia, PA); Kinako Abe (Union City, NJ); Yulia Barannikova (Brooklyn, NY)

Assignee: Plaid Inc.

G06F40/205G06F40/126G06F40/284G06N3/045

Abstract

Claims (64)

1. A computer-implemented method comprising, by one or more hardware processors executing program instructions:

receiving a text string including raw data for a transaction;

tokenizing the text string to create a sequence of tokens;

applying a masked language model to the sequence of tokens to generate a sequence of vectors, each of the vectors corresponding to one of the tokens and being encoded with information regarding one or more of surrounding tokens in the sequence of tokens,

wherein applying the masked language model comprises:

masking a token of the sequence of tokens,

sending the sequence of tokens including the masked token through an embedding layer to generate a 2D matrix,

encoding the 2D matrix with contextual information using a transformer encoder, and

applying a linear transformation to the encoded 2D matrix using a classification layer,

wherein the sequence of vectors are based on the linearly transformed encoded 2D matrix calculated for each of the tokens;

bidirectionally parsing the sequence of vectors to identify tokens indicative of entity information;

generating entity information using the tokens indicative of entity information; and

generating normalized transaction data including the generated entity information.

2. The computer-implemented method of claim 1 , wherein the masked language model comprises a neural network trained based on a corpus of raw transaction data.

3. The computer-implemented method of claim 1 , wherein the bidirectional parsing comprises applying a first long-short term memory (LSTM) neural network to the sequence of vectors in a first direction and a second LSTM neural network to the sequence of vectors in a second direction opposite to the first direction.

4. The computer-implemented method of claim 1 , wherein the sequence of vectors comprises a sequence of multi-dimensional numerical vectors.

5. The computer-implemented method of claim 1 , wherein the bidirectional parsing comprises:

generating a 2D matrix indicative of a type of each token in the text string based on the sequence of vectors, and

generating a label for each of the tokens in the text string based on the 2D matrix.

6. The computer-implemented method of claim 5 , wherein the label comprises a beginning entity token, an inside entity token, and an outside token.

7. The computer-implemented method of claim 6 , wherein the generating of the label for each of the tokens comprises a plurality of rules, including a first rule regarding a relative order of beginning entity token and any inside entity tokens, and a second rule regarding a number of beginning entity tokens.

8. The computer-implemented method of claim 1 , wherein the one or more hardware processors are associated with a multi-tenant system.

9. A system comprising:

one or more processors; and

a non-transitory computer readable memory having stored thereon instructions which, when executed by the one or more processors, cause the one or more processors to:

receive a text string including raw data for a transaction;

tokenize the text string to create a sequence of tokens;

apply a masked language model to the sequence of tokens to generate a sequence of vectors, each of the vectors corresponding to one of the tokens and being encoded with information regarding one or more of surrounding tokens in the sequence of tokens,

wherein applying the masked language model comprises:

masking a token of the sequence of tokens,

sending the sequence of tokens including the masked token through an embedding layer to generate a 2D matrix,

encoding the 2D matrix with contextual information using a transformer encoder, and

applying a linear transformation to the encoded 2D matrix using a classification layer,

 wherein the sequence of vectors are based on the linearly transformed encoded 2D matrix calculated for each of the tokens;

bidirectionally parse the sequence of vectors to identify tokens indicative of entity information;

generate entity information using the tokens indicative of entity information; and

generate normalized transaction data including the generated entity information.

10. The system of claim 9 , wherein the masked language model comprises a neural network trained based on a corpus of raw transaction data.

11. The system of claim 9 , wherein the bidirectional parsing comprises applying a first long-short term memory (LSTM) neural network to the sequence of vectors in a first direction and a second LSTM neural network to the sequence of vectors in a second direction opposite to the first direction.

12. The system of claim 9 , wherein the sequence of vectors comprises a sequence of multi-dimensional numerical vectors.

13. The system of claim 9 , wherein the bidirectional parsing comprises:

generating a 2D matrix indicative of a type of each token in the text string based on the sequence of vectors, and

generating a label for each of the tokens in the text string based on the 2D matrix.

14. The system of claim 9 , wherein the system is associated with a multi-tenant system.

15. A non-transitory computer readable medium having stored thereon instructions, which when executed by one or more processors, cause the one or more processors to:

receive a text string including raw data for a transaction;

tokenize the text string to create a sequence of tokens;

apply a masked language model to the sequence of tokens to generate a sequence of vectors, each of the vectors corresponding to one of the tokens and being encoded with information regarding one or more of surrounding tokens in the sequence of tokens,

wherein applying the masked language model comprises:

masking a token of the sequence of tokens,

sending the sequence of tokens including the masked token through an embedding layer to generate a 2D matrix,

encoding the 2D matrix with contextual information using a transformer encoder, and

applying a linear transformation to the encoded 2D matrix using a classification layer,

wherein the sequence of vectors are based on the linearly transformed encoded 2D matrix calculated for each of the tokens;

bidirectionally parse the sequence of vectors to identify tokens indicative of entity information;

generate entity information using the tokens indicative of entity information; and

generate normalized transaction data including the generated entity information.

16. The non-transitory computer readable medium of claim 15 , wherein the masked language model comprises a neural network trained based on a corpus of raw transaction data.

17. The non-transitory computer readable medium of claim 15 , wherein the bidirectional parsing comprises applying a first long-short term memory (LSTM) neural network to the sequence of vectors in a first direction and a second LSTM neural network to the sequence of vectors in a second direction opposite to the first direction.

18. The non-transitory computer readable medium of claim 15 , wherein the sequence of vectors comprises a sequence of multi-dimensional numerical vectors.

19. The non-transitory computer readable medium of claim 15 , wherein the bidirectional parsing comprises:

generating a 2D matrix indicative of a type of each token in the text string based on the sequence of vectors, and

generating a label for each of the tokens in the text string based on the 2D matrix.

20. The non-transitory computer readable medium of claim 15 , wherein the non-transitory computer readable medium is associated with a multi-tenant system.

Assignments (1)

Continuity (3)

References Cited (400)

Cited By (1)

US 12,718,222