Slot filling with contextual information

1. A system including an artificial neural network (ANN) for image editing, the system comprising:

a processor and a memory including computer program code, the memory and the computer program code configured to, with the processor, cause the system to convert text input into a set of word vectors;

a feature encoder of the ANN configured to create a combined feature vector for the text input based on the set of word vectors;

a scoring layer of the ANN configured to compute labeling scores based on the combined feature vector, wherein the feature encoder, the scoring layer, or both are trained using multi-task learning with a loss function including a first loss value and an additional loss value, wherein the additional loss value is based on mutual information, context-based prediction, or sentence-based prediction;

a command component configured to identify an image editing command based on the labeling scores; and

an image editing application configured to edit an image based on the image editing command.

2. The system of claim 1 , wherein:

the set of word vectors comprise semantic information and Part-of-Speech (POS) information.

3. The system of claim 1 , wherein the feature encoder further comprises:

a word encoder configured to compute a word-based feature vector; and

a structural encoder configured to compute a structure-based feature vector, wherein the combined feature vector comprises the word-based feature vector and the structure-based feature vector.

4. The system of claim 3 , wherein:

the word encoder comprises a 2-layer bi-directional Long Short Term Memory (LSTM).

5. The system of claim 3 , wherein:

the structural encoder comprises a Graph Convolutional Network (GCN).

6. The system of claim 1 , wherein the scoring layer further comprises:

a feed forward scoring layer configured to compute class scores based on the combined feature vector; and

a Conditional Random Field (CRF) scoring layer configured to compute the labeling scores based on the class scores.

7. A method for training a neural network, comprising:

receiving a training set comprising a plurality of text inputs and a set of target labels for each of the plurality of text inputs;

generating a combined feature vector for each of the plurality of text inputs;

generating a set of output labels for each of the combined feature vectors;

computing a first loss value by comparing the set of output labels and the set of target labels;

computing at least one additional loss value based on context information; and

updating the neural network based on the first loss value and the at least one additional loss value.

8. The method of claim 7 , further comprising:

embedding each word of the plurality of text inputs into a word embedding space;

identifying a Part-of-Speech (POS) tag for each of the words;

computing a word-based feature vector using a word encoder based on the embedded words and the identified POS tags;

computing a structure-based feature vector using a structural encoder; and

combining the word-based feature vector and context-based feature vector to produce the combined feature vector.

9. The method of claim 7 , further comprising:

computing class scores for each combined feature vector using a feed forward layer; and

applying a Conditional Random Field (CRF) to the class scores to compute the set of output labels.

10. The method of claim 9 , wherein:

the feed forward neural network comprises a 2 -layer feed forward neural network and a logistic regression model.

11. The method of claim 10 , wherein:

the first loss value is computed using a Viterbi loss algorithm.

12. The method of claim 7 , further comprising:

identifying a context-based feature vector for each word of a first subset of text inputs;

identifying a random vector for each word of a second subset of text inputs; and

training a binary mutual information classifier based on the context-based feature vectors for the each word of the first subset of text inputs and the random vectors,

wherein a mutual information loss value is computed using the binary mutual information classifier.

13. The method of claim 7 , further comprising:

training a feed forward neural network;

identifying context labels for the plurality of text inputs; and

computing the output labels using the feed forward neural network based on the context labels.

14. The method of claim 7 , further comprising:

computing a sentence vector for each text input of the plurality of text inputs by max pooling over words in the text input; and

applying a feed forward neural network to the sentence vectors to compute predicted label sets present in each of the text inputs, wherein each of the predicted label sets comprises a multi-label classification representing which labels are present in the corresponding text input.

15. The method of claim 7 , wherein the at least one additional loss value comprises:

a mutual information loss value based on comparing mutual information between the combined feature vectors and corresponding context-based feature vectors;

a context-based prediction loss value based on comparing context-based output labels and the target labels; or

a sentence-level prediction loss value based on comparing predicted label sets present in each of the text inputs with known label sets based on the target labels.

16. A method for image editing, comprising:

receiving an audible input for editing an image;

converting the audible input to a text input;

computing a feature vector for the text input using a neural network;

computing a set of word labels based on the feature vector using the neural network, wherein the neural network is trained using multi-task learning with a loss function including a first loss value and an additional loss value based on mutual information, context-based prediction, or sentence-based prediction;

identifying an image editing command based on the set of word labels; and

editing the image based on the image editing command.

17. The method of claim 16 , wherein:

the set of word labels includes an action label, an object label, an attribute label, or a value label.

18. The method of claim 16 , wherein:

the mutual information represents a loss value based on mutual information between word-based feature vectors and context-based feature vectors.

19. The method of claim 16 , wherein:

the context-based prediction represents a loss value based on max pooling over words in a text.

20. The method of claim 16 , wherein:

the sentence-based prediction represents a loss value based on a multi-label classification representing which labels are present in a text.