Method and system for intent classification

1. A computer-implemented method for determining an intent class of an input text, comprising:

providing a plurality of trained model sets, wherein each of the plurality of trained model sets is based on a different model and comprises a plurality of trained models, wherein each of the plurality trained models in the each of the plurality of trained model sets is obtained by training a same model with a different initialization condition over a pre-determined dataset;

feeding the input text into each of the plurality trained models in the each of the plurality of trained model sets to thereby obtain a plurality of prediction results, each indicating a likely intent class of the input text, wherein the plurality of prediction results correspond respectively to the plurality of trained models in each of the plurality of trained model sets;

ensembling prediction results corresponding to a same trained model set for each of the plurality of trained model sets to thereby obtain a plurality of first-layer ensembles corresponding respectively to the plurality of trained model sets;

ensembling the plurality of first-layer ensembles to thereby obtain at least one second-layer ensemble, wherein each of the at least one second-layer ensemble is obtained by ensembling two or more of the plurality of first-layer ensembles; and

determining the intent class of the input text based on the at least one second-layer ensemble;

the method further comprising:

evaluating each of the plurality of first-layer ensembles and each of the plurality of second-layer ensembles using pre-determined labels from a test set of the pre-determined dataset; and

comparing a metric corresponding to each of the plurality of first-layer ensembles and each of the plurality of second-layer ensembles, and identifying the one or the combination of the models for the intent classification method if the one or the combination of the models implicated in one of the plurality of first-layer ensembles or the plurality of second-layer ensembles has a most favorable metric compared with others.

2. The computer-implemented method of claim 1 , wherein the providing a plurality of trained model sets comprises:

providing a plurality of models; and

training each of the plurality of models using data from the pre-determined dataset to thereby obtain the plurality of trained model sets.

3. The computer-implemented method of claim 2 , wherein the plurality of models comprise at least one neural networks-based model.

4. The computer-implemented method of claim 3 , wherein each trained model in each trained model set that is based on a neural networks-based model is trained with different random initializations of weights.

5. The computer-implemented method of claim 3 , wherein the at least one neural networks-based model comprises one or more convolutional neural networks-based models, each selected from CNN, CharCNN, ABiCNN, or ACharBiCNN.

6. The computer-implemented method of claim 5 , wherein the one or more convolutional neural networks-based models comprises ABiCNN.

7. The computer-implemented method of claim 3 , wherein the at least one neural networks-based model comprises one or more recurrent neural networks-based models, each selected from GRU, CharGRU, BiGRU, and CharBiGRU, LSTM, CharLSTM, BiLSTM, or CharBiLSTM.

8. The computer-implemented method of claim 3 , wherein the at least one neural networks-based model comprises at least one convolutional neural networks-based model and at least one recurrent neural networks-based model.

9. The computer-implemented method of claim 3 , wherein the pre-determined dataset is specific to a language whose characters contain rich semantic information, wherein each of the at least one neural networks-based model has character-level embeddings.

10. The computer-implemented method of claim 2 , wherein the pre-determined dataset is ATIS, and the plurality of models comprise GRU, BiGRU, BiLSTM, CNN, CharCNN, ABiCNN, and ACharBiCNN.

11. The computer-implemented method of claim 2 , wherein the pre-determined dataset is Banking, and the plurality of models comprise CharGRU, CharLSTM, CharBiGRU, CharBiLSTM, CNN, CharCNN, ABiCNN, and ACharBiCNN.

12. The computer-implemented method of claim 2 , wherein the pre-determined dataset is SMP, and the plurality of models comprise CharLSTM, CharBiGRU, CharBiLSTM, CharCNN, ABiCNN, and ACharBiCNN.

13. The computer-implemented method of claim 1 , wherein one or both of the ensembling prediction results corresponding to a same trained model set for each of the plurality of trained model sets to thereby obtain a plurality of first-layer ensembles corresponding respectively to the plurality of trained model sets and the ensembling the plurality of first-layer ensembles to thereby obtain at least one second-layer ensemble, wherein each of the at least one second-layer ensemble is obtained by ensembling two or more of the plurality of first-layer ensembles is performed under

a majority vote with confidence approach.

14. A system for determining an intent class of an input text, comprising:

a computer-readable memory storing executable instructions; and

one or more processors in communication with the computer-readable memory, wherein the one or more processors are programmed by the executable instructions to at least:

feed the input text into each of a plurality trained models in each of a plurality of trained model sets to thereby calculate a plurality of prediction results, each indicating a likely intent class of the input text, wherein the plurality of prediction results correspond respectively to the plurality of trained models in the each of the plurality of trained model sets;

ensemble prediction results corresponding to a same trained model set for each of the plurality of trained model sets to thereby obtain a plurality of first-layer ensembles corresponding respectively to the plurality of trained model sets; and

ensemble the plurality of first-layer ensembles to thereby obtain at least one second-layer ensemble, wherein each of the at least one second-layer ensemble is obtained by ensembling two or more of the plurality of first-layer ensembles;

wherein the one or more processors are further configured to:

evaluate each of the plurality of first-layer ensembles and each of the plurality of second-layer ensembles using pre-determined labels from a test set of the pre-determined dataset; and

compare a metric corresponding to each of the plurality of first-layer ensembles and each of the plurality of second-layer ensembles, and identifying the one or the combination of the models for the intent classification method if the one or the combination of the models implicated in one of the plurality of first-layer ensembles or the plurality of second-layer ensembles has a most favorable metric compared with others.

15. The system of claim 14 , wherein the one or more processors are programmed by the executable instructions to further:

train each of a plurality of models using data from a pre-determined dataset to thereby obtain the plurality of trained model sets, wherein each of the plurality of trained model sets is based on a different model, and each of the plurality trained models in the each of the plurality of trained model sets is based on a same model with a different initialization condition.

16. The system of claim 15 , wherein each of the plurality of models is a neural networks-based model, and the different initialization condition comprises different random initialization of weights.

17. The system of claim 16 , wherein the neural networks-based model is a convolutional neural networks-based model or a recurrent neural networks-based model.

18. The system of claim 15 , wherein the one or more processors are programmed by the executable instructions to further:

convert an input data into the input text.

19. A computer-implemented process for identifying one or a combination of models for an intent classification method with favorable accuracy based on a dataset, comprising:

providing a plurality of models;

training each of the plurality of models using data from a train set of the dataset to thereby obtain a plurality of trained model sets, wherein each of the plurality of trained model sets is based on a different model and comprises a plurality of trained models, wherein each of the plurality trained models in the each of the plurality of trained model sets is obtained by training a same model with a different initialization condition over the dataset;

feeding an input text in a test set of the dataset into each of the plurality trained models in the each of the plurality of trained model sets to thereby obtain a plurality of prediction results, each indicating a likely intent class of the input text, wherein the plurality of prediction results correspond respectively to the plurality of trained models in each of the plurality of trained model sets;

ensembling prediction results corresponding to a same trained model set for each of the plurality of trained model sets to thereby obtain a plurality of first-layer ensembles corresponding respectively to the plurality of trained model sets;

ensembling the plurality of first-layer ensembles to thereby obtain a plurality of second-layer ensembles, wherein each of the plurality of second-layer ensembles is obtained by ensembling one of all possible combinations among the plurality of first-layer ensembles;

evaluating each of the plurality of first-layer ensembles and each of the plurality of second-layer ensembles using pre-determined labels from the test set; and

comparing a metric corresponding to each of the plurality of first-layer ensembles and each of the plurality of second-layer ensembles, and identifying the one or the combination of the models for the intent classification method if the one or the combination of the models implicated in one of the plurality of first-layer ensembles or the plurality of second-layer ensembles has a most favorable metric compared with others.

20. The computer-implemented process of claim 19 , wherein the metric comprises an unweighted F1 score.