Language sequence labeling method and apparatus, storage medium, and computing device

1. A language sequence labeling method, performed by a computing device, comprising:

reading a first embedding representation of a language sequence, the first embedding representation comprising a character-level word embedding representation, a pre-trained word embedding representation, and a global word embedding representation of the language sequence, the global word embedding representation referring to a global context representation of the language sequence;

performing first depth transformation (DT) encoding on the first embedding representation based on a first DT recurrent neural network (RNN), to output a first hidden-layer state representation corresponding to each word in the language sequence; and

decoding the first hidden-layer state representations of the language sequence, to obtain a labeling result of one or more elements in the language sequence.

2. The method according to claim 1 , further comprising:

reading a second embedding representation of the language sequence, the second embedding representation comprising the character-level word embedding representation and the pre-trained word embedding representation; and

performing second DT encoding on the second embedding representation based on a second DT RNN, to obtain the global word embedding representation.

3. The method according to claim 2 , further comprising:

performing information aggregation on a result obtained after the second DT encoding, to obtain the global word embedding representation.

4. The method according to claim 1 , wherein the performing first DT encoding on the first embedding representation based on a first DT RNN, to output a first hidden-layer state representation corresponding to each word in the language sequence comprises:

performing the first DT encoding on the first embedding representation in forward and reverse directions respectively; and

concatenating DT encoding results obtained in the forward and reverse directions, to obtain the first hidden-layer state representation corresponding to the each word.

5. The method according to claim 1 , further comprising:

constructing the first DT RNN by using a linear transformation enhanced gated recurrent unit (L-GRU) and a transition gated recurrent unit (T-GRU).

6. The method according to claim 5 , wherein the first DT RNN comprises one layer of L-GRUs and at least two layers of T-GRUs.

7. The method according to claim 1 , wherein the decoding the first hidden-layer state representation, to obtain a labeling result of one or more elements in the language sequence comprises:

performing, for the each word, DT on the first hidden-layer state representation of the word based on a third DT RNN, to obtain a second hidden-layer state representation; and

obtaining the labeling result based on the second hidden-layer state representation.

8. The method according to claim 7 , wherein the performing DT on the first hidden-layer state representation of the word comprises: performing unidirectional DT.

9. The method according to claim 7 , wherein the obtaining the labeling result of the language sequence based on the second hidden-layer state representation comprises:

presetting a plurality of labels; and

performing linear transformation on the second hidden-layer state representation and label information at a previous moment, to obtain a probability that the word belongs to each label.

10. The method according to claim 3 , wherein the information aggregation comprises average pooling or maximum pooling.

11. A language sequence labeling apparatus, comprising a processor and a memory storing a computer program, the computer program being configured to, when executed on the processor, cause the processor to:

read a first embedding representation of a language sequence, the first embedding representation comprising a character-level word embedding representation, a pre-trained word embedding representation, and a global word embedding representation of the language sequence, the global word embedding representation referring to a global context representation of the language sequence;

perform first DT encoding on the first embedding representation based on a first DT recurrent neural network (RNN), to output a first hidden-layer state representation corresponding to each word in the language sequence; and

decode the first hidden-layer state representations of the language sequence, to obtain a labeling result of one or more elements in the language sequence.

12. The apparatus according to claim 11 , wherein the computer program further cause the processor to:

obtain the global word embedding representation, and comprising:

read a second embedding representation of the language sequence, the second embedding representation comprising the character-level word embedding representation and the pre-trained word embedding representation; and

perform second DT encoding on the second embedding representation based on a second DT RNN, to obtain the global word embedding representation.

13. The apparatus according to claim 12 , wherein the computer program further cause the processor to:

perform information aggregation on a result obtained after the second DT encoding, to obtain the global word embedding representation.

14. The apparatus according to claim 11 , wherein the computer program further cause the processor to perform the first DT encoding on the first embedding representation in forward and reverse directions respectively; and concatenate DT encoding results obtained in the forward and reverse directions, to obtain the first hidden-layer state representation corresponding to the each word.

15. The apparatus according to claim 11 , wherein the computer program further cause the processor to:

construct the first DT RNN by using a linear transformation enhanced gated recurrent unit (L-GRU) and a transition gated recurrent unit (T-GRU).

16. The apparatus according to claim 15 , wherein the first DT RNN comprises one layer of L-GRUs and at least two layers of T-GRUs.

17. The apparatus according to claim 11 , wherein the computer program further cause the processor to:

perform, for the each word, DT on the first hidden-layer state representation of the word based on a third DT RNN, to obtain a second hidden-layer state representation; and

obtain the labeling result based on the second hidden-layer state representation.

18. The apparatus according to claim 17 , wherein the computer program further cause the processor to:

preset a plurality of labels, wherein

perform linear transformation on the second hidden-layer state representation and label information at a previous moment, to obtain a probability that the word belongs to each label.

19. The apparatus according to claim 13 , wherein the information aggregation comprises average pooling or maximum pooling.

20. A non-transitory computer-readable storage medium, storing computer program instructions, the computer program instructions, when executed by a processor, causing the processor to perform:

reading a first embedding representation of a language sequence, the first embedding representation comprising a character-level word embedding representation, a pre-trained word embedding representation, and a global word embedding representation of the language sequence, the global word embedding representation referring to a global context representation of the language sequence;

performing first depth transformation (DT) encoding on the first embedding representation based on a first DT recurrent neural network (RNN), to output a first hidden-layer state representation corresponding to each word in the language sequence; and

decoding the first hidden-layer state representations of the language sequence, to obtain a labeling result of one or more elements in the language sequence.