Systems and methods for training models

1 . A computer-implemented method comprising:

storing, by a computer-readable memory, a training data database comprising a plurality of datasets of example documents generated by or associated with customers of an accounting system;

at a model training system of the accounting system, the model training system comprising a network interface, one or more processors, and a computer-readable memory storing a model training module, and models to be trained comprising an image-based numerical representation generation model and a text-based numerical representation generation model, the model training system further comprising one or more processors configured to execute instructions stored in the memory, extracting from the training data database and via a communications network and the network interface, a batch of training data for training models to be trained, the extracted training data comprising a plurality of datasets, each dataset associated with a label and comprising at least one numerical representation of an example document;

determining, by the model training system, a number of classes of labels in the batch, wherein each class is associated with a unique attribute value;

determining, by the model training system, a number of numerical representations associated with each class in the batch;

providing the numerical representations to the models to be trained;

for each numerical representation in each dataset:

determining, by the models to be trained, a first similarity measure indicative of a similarity of the numerical representation to other first numerical representations associated with a same class;

determining, by the models to be trained, a second similarity measure for each of other datasets associated with a different respective class in the batch, each second similarity measure indicative of the similarity of the numerical representation to each of the at least one numerical representations of respective other datasets associated with respective different classes of the batch;

determining, by the models to be trained, a difference measure as a function of the first similarity measure and one or more second similarity measures; and

determining, by the models to be trained, a normalized difference measure by dividing the difference measure by the number of example documents associated with the same class of the datasets;

determining, by the model training system, a loss value as a function of normalized difference measures of the example documents in the batch, wherein determining the loss value comprises summing the normalized difference measures of the numerical representations in the batch and dividing by the number of classes that have a dataset with at least one numerical representation;

back-propagating the loss value through a plurality of layers of the models being trained including calculating, by the model training system, based on the determined loss value, at each layer of neurons in the text-based numerical representation generation model and the image-based numerical representation generation model a plurality of error gradients; and

adjusting, by the model training system, based on the plurality of error gradients, weights of each layer of neurons in the text-based numerical representation generation model and the image-based numerical representation generation model to train the text-based numerical representation generation model and the image-based numerical representation generation model.

2 . The method of claim 1 , wherein determining the second similarity measure for each of the other datasets associated with a different respective class in the batch comprises:

determining, by the models to be trained, a second similarity measure for each of the other datasets; and

disregarding or ignoring a second similarity measure for each other dataset associated with a class corresponding to the class of the datasets.

3 . The method of claim 1 , wherein the difference measure is indicative of the similarity of the example document to other example documents associated with the same class relative to the example documents of the other datasets associated with the respective different classes of the batch.

4 . The method of claim 1 , wherein determining the first similarity measure comprises determining, by the models to be trained, an average dot product of the numerical representation to each other numerical representations associated with the same class, and wherein determining the second similarity measure comprises determining, by the models to be trained, the average dot product of the numerical representation to each of the other numerical representations in other respective datasets associated with the respective different classes.

5 . The method of claim 1 , wherein the labels are entity labels and each class of label identifies a particular entity.

6 . The method of claim 1 , wherein the example documents are derived from previously reconciled accounting documents of the accounting system, each of which has been associated with a respective entity, and wherein the label of each document is indicative of the respective entity.

7 . A computing system comprising:

a computer-readable memory configured to store a training data database comprising a plurality of datasets of example documents generated by or associated with customers of an accounting system; and

the accounting system comprising a model training system, the model training system comprising:

a network interface;

one or more processors; and

a computer-readable memory storing a model training module, and models to be trained comprising an image-based numerical representation generation model and a text-based numerical representation generation model, the computer-readable memory further storing computer executable instructions, which when executed by the one or more processors, cause the one or more processors to:

extract from the training data database and via a communications network and the network interface, a batch of training data for training the models to be trained, the extracted training data comprising a plurality of datasets, each dataset associated with a label and comprising at least one numerical representation of an example document;

determine a number of classes of labels in the batch, wherein each class is associated with a unique attribute value;

determine a number of numerical representations associated with each class in the batch;

provide the numerical representations to the models to be trained;

for each numerical representation in each dataset:

determine, by the models to be trained, a first similarity measure indicative of a similarity of the numerical representation to other first numerical representations associated with a same class;

determine, by the models to be trained, a second similarity measure for each of other datasets associated with a different respective class in the batch, each second similarity measure indicative of the similarity of the numerical representation to each of the at least one numerical representations of respective other datasets associated with respective different classes of the batch;

determine, by the models to be trained, a difference measure as a function of the first similarity measure and one or more second similarity measures; and

determine, by the models to be trained, a normalized difference measure by dividing the difference measure by the number of example documents associated with the same class of the datasets;

determine, by the model training system, a loss value as a function of normalized difference measures of the example documents in the batch, wherein determining the loss value comprises summing the normalized difference measures of the numerical representations in the batch and dividing by the number of classes that have a dataset with at least one numerical representation; and

back-propagate, by the model training system, the loss value through a plurality of layers of the models being trained including determining, based on the determined loss value, at each layer of neurons in the text-based numerical representation generation model and the image-based numerical representation generation model a plurality of error gradients; and

adjust, by the model training system, based on the plurality of error gradients, weights of each layer of neurons in the text-based numerical representation generation model and the image-based numerical representation generation model to train the text-based numerical representation generation model and the image-based numerical representation generation model.

8 . The computing system of claim 7 , wherein determining the second similarity measure for each of the other datasets associated with a different respective class in the batch comprises:

determining, by the models to be trained, a second similarity measure for each of the other datasets; and

disregarding or ignoring a second similarity measure for each other dataset associated with a class corresponding to the class of the datasets.

9 . The computing system of claim 7 , wherein the difference measure is indicative of the similarity of the example document to other example documents associated with the same class relative to the example documents of the other datasets associated with the respective different classes of the batch.

10 . The computing system of claim 7 , wherein determining the first similarity measure comprises determining, by the models to be trained, an average dot product of the numerical representation to each other numerical representations associated with the same class, and wherein determining the second similarity measure comprises determining, by the models to be trained, the average dot product of the numerical representation to each of the other numerical representations in other respective datasets associated with the respective different classes.

11 . A computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform operations including:

storing, by a computer-readable memory, a training data database comprising a plurality of datasets of example documents generated by or associated with customers of an accounting system;

at a model training system of the accounting system, the model training system comprising a network interface, one or more processors, and a computer-readable memory storing a model training module, and models to be trained comprising an image-based numerical representation generation model and a text-based numerical representation generation model, the model training system further comprising one or more processors configured to execute instructions stored in the memory, extracting from the training data database and via a communications network and the network interface, a batch of training data for training the models to be trained, the extracted training data comprising a plurality of datasets, each dataset associated with a label and comprising at least one numerical representation of an example document;

determining, by the model training system, a number of classes of labels in the batch, wherein each class is associated with a unique attribute value;

determining, by the model training system, a number of numerical representations associated with each class in the batch;

providing the numerical representations to the models to be trained;

for each numerical representation in each dataset:

determining, by the models to be trained, a first similarity measure indicative of a similarity of the numerical representation to other first numerical representations associated with a same class;

determining, by the models to be trained, a second similarity measure for each of other datasets associated with a different respective class in the batch, each second similarity measure indicative of the similarity of the numerical representation to each of the at least one numerical representations of respective other datasets associated with respective different classes of the batch;

determining, by the models to be trained, a difference measure as a function of the first similarity measure and one or more second similarity measures; and

determining, by the models to be trained, a normalized difference measure by dividing the difference measure by the number of example documents associated with the same class of the datasets;

determining, by the model training system, a loss value as a function of normalized difference measures of the example documents in the batch, wherein determining the loss value comprises summing the normalized difference measures of the numerical representations in the batch and dividing by the number of classes that have a dataset with at least one numerical representation;

back-propagating the loss value through a plurality of layers of the models being trained including calculating, by the accounting system, based on the determined loss value, at each layer of neurons in the text-based numerical representation generation model and the image-based numerical representation generation model a plurality of error gradients; and

adjusting, by the model training system, based on the plurality of error gradients, weights of each layer of neurons in the text-based numerical representation generation model and the image-based numerical representation generation model to train the text-based numerical representation generation model and the image-based numerical representation generation model.

12 . The computer-readable storage medium of claim 11 , wherein determining the second similarity measure for each of the other datasets associated with a different respective class in the batch comprises:

determining, by the models to be trained, a second similarity measure for each of the other datasets; and

disregarding or ignoring a second similarity measure for each other dataset associated with a class corresponding to the class of the datasets.

13 . The computer-readable storage medium of claim 11 , wherein the difference measure is indicative of the similarity of the example document to other example documents associated with the same class relative to the example documents of the other datasets associated with the respective different classes of the batch.

14 . The computer-readable storage medium of claim 11 , wherein determining the first similarity measure comprises determining, by the models to be trained, an average dot product of the numerical representation to each other numerical representations associated with the same class, and wherein determining the second similarity measure comprises determining, by the models to be trained, the average dot product of the numerical representation to each of the other numerical representations in other respective datasets associated with the respective different classes.