Method, apparatus and storage medium for recognizing character
A method for recognizing a character includes: obtaining a character; converting the character into a radical based character recognition result, where the radical based character recognition result comprises symbols indicating radicals of the character and a structure of the radicals of the character; and recognizing the character based on the radical based character recognition result.
1. A method for recognizing a character, comprising:
obtaining a character;
converting the character into a radical based character recognition result, wherein the radical based character recognition result comprises symbols indicating radicals of the character and a structure of the radicals of the character; and
recognizing the character based on the radical based character recognition result,
wherein the radical based character recognition result is in a form of a structure identity {radical . . . radical},
wherein the structure identity comprises a symbol indicating a left-right structure, a symbol indicating a top-bottom structure, a symbol indicating a top-left-surround structure, a symbol indicating a top-right-surround structure, a symbol indicating a bottom-left-surround structure, a symbol indicating a left-surround structure, a symbol indicating a bottom-surround structure, a symbol indicating a top-surround structure, a symbol indicating a surround structure, or a symbol indicating a within structure.
2. The method according to claim 1 , wherein converting the character into the radical based character recognition result comprises: inputting the character into a character recognition model to obtain the radical based character recognition result, wherein the character recognition model is obtained by training on a set of samples, and wherein the character recognition model comprises an encoder and a decoder.
3. The method according to claim 2 , wherein inputting the character into the character recognition model comprises inputting data of an image including the character into the encoder of the character recognition model, and the encoder comprises a convolutional neural network.
4. The method according to claim 3 , wherein
the encoder comprises a full convolutional neural network, the full convolutional neural network comprises an input layer, a first predetermined quantity of convolutional layers, a second predetermined quantity of pooling layers, and an output layer,
wherein the input layer comprises a plurality of input neurons for describing pixels of the image, the output layer comprises an output neuron for describing annotation vectors of the image, the convolutional layers each performs feature extraction on an output of a previous layer by using at least one predetermined convolution kernel to obtain an output of the convolutional layer itself, and the pooling layers each performs aggregation statistical processing on an output of a previous convolutional layer by using at least one predetermined pooling kernel to obtain an output of the pooling layer itself.
5. The method according to claim 2 , wherein inputting the character into the character recognition model comprises obtaining data of the character with stroke information and inputting the data of the character with the stroke information into the encoder of the character recognition model, wherein the stroke information comprises pen-down or pen-up information of each point of the character, and wherein the encoder comprises a recurrent neural network (RNN).
6. The method according to claim 5 , wherein the encoder comprises a predetermined number of stacked bidirectional Gated Recurrent Unit (GRU) layers,
wherein the first bidirectional GRU layer generates a current GRU hidden state of the first bidirectional GRU layer based on the data of the character with the stroke information, a previous GRU hidden state of the first bidirectional GRU layer and a next GRU hidden state of the first bidirectional GRU layer, and
each of other bidirectional GRU layers except for the first bidirectional GRU layer generates a current GRU hidden state of the bidirectional GRU layer itself based on a current GRU hidden state of a previous bidirectional GRU layer outputted from the previous bidirectional GRU layer, a previous GRU hidden state of the bidirectional GRU layer itself and a next GRU hidden state of the bidirectional GRU layer itself,
wherein the current GRU hidden state of the last bidirectional GRU layer is used as annotation vectors of the character, to be inputted into the decoder.
7. The method according to claim 5 , wherein the encoder comprises a predetermined number of stacked bidirectional GRU layers and a pooling layer,
wherein the first bidirectional GRU layer generates a current GRU hidden state of the first bidirectional GRU layer based on the data of the character with the stroke information, a previous GRU hidden state of the first bidirectional GRU layer and a next GRU hidden state of the first bidirectional GRU layer,
each of other bidirectional GRU layers except for the first bidirectional GRU layer generates a current GRU hidden state of the bidirectional GRU layer itself based on a current GRU hidden state of a previous bidirectional GRU layer outputted from the previous bidirectional GRU layer, a previous GRU hidden state of the bidirectional GRU layer itself and a next GRU hidden state of the bidirectional GRU layer itself, and
the pooling layer generates annotation vectors of the character, to be inputted into the decoder, based on the current GRU hidden state of the last bidirectional GRU layer.
8. The method according to claim 2 , wherein
the decoder comprises a first unidirectional GRU layer, an attention layer and a second unidirectional GRU layer,
wherein the first unidirectional GRU layer generates a prediction of a current GRU hidden state of the first unidirectional GRU layer based on a previous GRU hidden state of the second unidirectional GRU layer outputted from the second unidirectional GRU layer and a previous symbol,
wherein the attention layer generates a context vector based on the prediction of the current GRU hidden state of the first unidirectional GRU layer outputted from the first unidirectional GRU layer and annotation vectors outputted from the encoder,
wherein the second unidirectional GRU layer generates a current GRU hidden state of the second unidirectional GRU layer based on the context vector outputted from the attention layer and the prediction of the current GRU hidden state of the first unidirectional GRU layer outputted from the first unidirectional GRU layer,
wherein the decoder further generates a probability indicating whether an input is a target symbol based on the context vector, the current GRU hidden state of the second unidirectional GRU layer and the previous symbol.
9. The method according to claim 8 , wherein the attention layer comprises a convolutional layer and a plurality of fully connected layers,
wherein the convolutional layer generates a coverage vector based on a previous attention coefficient matrix,
the plurality of fully connected layers generate a current attention coefficient matrix based on the coverage vector outputted from the convolutional layer, the annotation vectors outputted from the encoder and the prediction of the current GRU hidden state of the first unidirectional GRU layer outputted from the first unidirectional GRU layer,
the attention layer further generates the context vector based on the current attention coefficient matrix and the annotation vectors outputted from the encoder.
10. A non-transitory storage medium storing a computer program, wherein the computer program comprises a plurality of computer-readable instructions that, when executed by a computer, cause the computer to:
obtain a character;
convert the character into a radical based character recognition result, wherein the radical based character recognition result comprises symbols indicating radicals of the character and a structure of the radicals of the character; and
recognize the character based on the radical based character recognition result,
wherein the radical based character recognition result is in a form of a structure identity {radical . . . radical},
wherein the structure identity comprises a symbol indicating a left-right structure, a symbol indicating a top-bottom structure, a symbol indicating a top-left-surround structure, a symbol indicating a top-right-surround structure, a symbol indicating a bottom-left-surround structure, a symbol indicating a left-surround structure, a symbol indicating a bottom-surround structure, a symbol indicating a top-surround structure, a symbol indicating a surround structure, or a symbol indicating a within structure.
11. An apparatus for recognizing a character, comprising a processor and a non-transitory storage medium having a plurality of instructions stored thereon that, when executed by the processor, cause the processor to:
obtain a character;
convert the character into a radical based character recognition result, wherein the radical based character recognition result comprises symbols indicating radicals of the character and a structure of the radicals of the character; and
recognize the character based on the radical based character recognition result,
wherein the radical based character recognition result is in a form of a structure identity {radical . . . radical},
wherein the structure identity comprises a symbol indicating a left-right structure, a symbol indicating a top-bottom structure, a symbol indicating a top-left-surround structure, a symbol indicating a top-right-surround structure, a symbol indicating a bottom-left-surround structure, a symbol indicating a left-surround structure, a symbol indicating a bottom-surround structure, a symbol indicating a top-surround structure, a symbol indicating a surround structure, or a symbol indicating a within structure.
12. The apparatus according to claim 11 , wherein converting the character into the radical based character recognition result comprises: inputting the character into a character recognition model to obtain the radical based character recognition result, wherein the character recognition model is obtained by training on a set of samples, and wherein the character recognition model comprises an encoder and a decoder.
13. The apparatus according to claim 12 , wherein inputting the character into the character recognition model comprises inputting data of an image including the character into the encoder of the character recognition model, and the encoder comprises a convolutional neural network.
14. The apparatus according to claim 13 , wherein
the encoder comprises a full convolutional neural network, the full convolutional neural network comprises an input layer, a first predetermined quantity of convolutional layers, a second predetermined quantity of pooling layers, and an output layer,
wherein the input layer comprises a plurality of input neurons for describing pixels of the image, the output layer comprises an output neuron for describing annotation vectors of the image, the convolutional layers each performs feature extraction on an output of a previous layer by using at least one predetermined convolution kernel to obtain an output of the convolutional layer itself, and the pooling layers each performs aggregation statistical processing on an output of a previous convolutional layer by using at least one predetermined pooling kernel to obtain an output of the pooling layer itself.
15. The apparatus according to claim 12 , wherein inputting the character into the character recognition model comprises obtaining data of the character with stroke information and inputting the data of the character with the stroke information into the encoder of the character recognition model, wherein the stroke information comprises pen-down or pen-up information of each point of the character, and wherein the encoder comprises a recurrent neural network (RNN).
16. The apparatus according to claim 15 , wherein the encoder comprises a predetermined number of stacked bidirectional GRU layers and a pooling layer,
wherein the first bidirectional GRU layer generates a current GRU hidden state of the first bidirectional GRU layer based on the data of the character with the stroke information, a previous GRU hidden state of the first bidirectional GRU layer and a next GRU hidden state of the first bidirectional GRU layer,
each of other bidirectional GRU layers except for the first bidirectional GRU layer generates a current GRU hidden state of the bidirectional GRU layer itself based on a current GRU hidden state of a previous bidirectional GRU layer outputted from the previous bidirectional GRU layer, a previous GRU hidden state of the bidirectional GRU layer itself and a next GRU hidden state of the bidirectional GRU layer itself, and
the pooling layer generates annotation vectors of the character, to be inputted into the decoder, based on the current GRU hidden state of the last bidirectional GRU layer.
17. The apparatus according to claim 12 , wherein
the decoder comprises a first unidirectional GRU layer, an attention layer and a second unidirectional GRU layer,
wherein the first unidirectional GRU layer generates a prediction of a current GRU hidden state of the first unidirectional GRU layer based on a previous GRU hidden state of the second unidirectional GRU layer outputted from the second unidirectional GRU layer and a previous symbol,
wherein the attention layer generates a context vector based on the prediction of the current GRU hidden state of the first unidirectional GRU layer outputted from the first unidirectional GRU layer and annotation vectors outputted from the encoder,
wherein the second unidirectional GRU layer generates a current GRU hidden state of the second unidirectional GRU layer based on the context vector outputted from the attention layer and the prediction of the current GRU hidden state of the first unidirectional GRU layer outputted from the first unidirectional GRU layer,
wherein the decoder further generates a probability indicating whether an input is a target symbol based on the context vector, the current GRU hidden state of the second unidirectional GRU layer and the previous symbol.
18. The apparatus according to claim 17 , wherein the attention layer comprises a convolutional layer and a plurality of fully connected layers,
wherein the convolutional layer generates a coverage vector based on a previous attention coefficient matrix,
the plurality of fully connected layers generate a current attention coefficient matrix based on the coverage vector outputted from the convolutional layer, the annotation vectors outputted from the encoder and the prediction of the current GRU hidden state of the first unidirectional GRU layer outputted from the first unidirectional GRU layer,
the attention layer further generates the context vector based on the current attention coefficient matrix and the annotation vectors outputted from the encoder.