Managing real-time handwriting recognition
Methods, systems, and computer-readable media related to a technique for providing handwriting input functionality on a user device. A handwriting recognition module is trained to have a repertoire comprising multiple non-overlapping scripts and capable of recognizing tens of thousands of characters using a single handwriting recognition model. The handwriting input module provides real-time, stroke-order and stroke-direction independent handwriting recognition for multi-character handwriting input. In particular, real-time, stroke-order and stroke-direction independent handwriting recognition is provided for multi-character, or sentence level Chinese handwriting recognition. User interfaces for providing the handwriting input functionality are also disclosed.
1. A method, comprising:
at a device with a display and a touch-sensitive surface:
receiving, via the touch-sensitive surface, a handwriting input from a user, the handwriting input comprising a plurality of handwritten strokes provided in a handwriting input area of a handwriting input interface;
recognizing, based on a handwriting recognition model, a plurality of output characters from the handwriting input, the plurality of output characters including at least a first emoji character and at least a first character from a script of a natural human language; and
displaying, on the display, a first recognition result comprising the first emoji character and the first character from the script of the natural human language in a candidate display area of the handwriting input interface.
2. The method of claim 1 , further comprising:
based on the handwriting recognition model, recognizing at least a first semantic unit from the handwriting input, wherein the first semantic unit comprises a respective character, word or phrase that is capable of conveying a respective semantic meaning in a respective human language;
identifying a second emoji character associated with the first semantic unit recognized from the handwriting input; and
displaying, in the candidate display area of the handwriting input interface, a second recognition result comprising at least the second emoji character identified from the first semantic unit.
3. The method of claim 2 , wherein displaying the second recognition result further comprises:
concurrently displaying the second recognition result with a third recognition result comprising at least the first semantic unit.
4. The method of claim 1 , further comprising:
receiving a user input selecting the first recognition result displayed in the candidate display area; and
in response to the user input, entering text of the selected first recognition result in a text input area of the handwriting input interface, wherein the text includes at least the first emoji character and the first character from the script of the natural human language.
5. The method of claim 1 , wherein the handwriting recognition model has been trained on a multi-script training corpus comprising writing samples corresponding to characters of at least three non-overlapping scripts, and the three non-overlapping scripts includes a set of emoji characters, Chinese characters, and Latin script.
6. The method of claim 1 , further comprising:
identifying a second semantic unit corresponding to the first emoji character recognized from the handwriting input;
displaying, in the candidate display area of the handwriting input interface, a fourth recognition result comprising at least the second semantic unit identified from the first emoji character.
7. The method of claim 6 , wherein displaying the fourth recognition result further comprises:
concurrently displaying the fourth recognition result with the first recognition result in the candidate display area.
8. A non-transitory computer-readable medium storing one or more programs configured to be executed by one or more processors of an electronic device with a display and a touch-sensitive surface, the one or more programs including instructions for:
receiving, via the touch-sensitive surface, a handwriting input from a user, the handwriting input comprising a plurality of handwritten strokes provided in a handwriting input area of a handwriting input interface;
recognizing, based on a handwriting recognition model, a plurality of output characters from the handwriting input, the plurality of output characters including at least a first emoji character and at least a first character from a script of a natural human language; and
displaying, on the display, a first recognition result comprising the first emoji character and the first character from the script of the natural human language in a candidate display area of the handwriting input interface.
9. The non-transitory computer-readable medium of claim 8 , the one or more programs further including instructions for:
based on the handwriting recognition model, recognizing at least a first semantic unit from the handwriting input, wherein the first semantic unit comprises a respective character, word or phrase that is capable of conveying a respective semantic meaning in a respective human language;
identifying a second emoji character associated with the first semantic unit recognized from the handwriting input; and
displaying, in the candidate display area of the handwriting input interface, a second recognition result comprising at least the second emoji character identified from the first semantic unit.
10. The non-transitory computer-readable medium of claim 9 , wherein displaying the second recognition result further comprises:
concurrently displaying, on the display, the second recognition result with a third recognition result comprising at least the first semantic unit.
11. The non-transitory computer-readable medium of claim 8 , the one or more programs further including instructions for:
receiving a user input selecting the first recognition result displayed in the candidate display area; and
in response to the user input, entering text of the selected first recognition result in a text input area of the handwriting input interface, wherein the text includes at least the first emoji character and the first character from the script of the natural human language.
12. The non-transitory computer-readable medium of claim 8 , wherein the handwriting recognition model has been trained on a multi-script training corpus comprising writing samples corresponding to characters of at least three non-overlapping scripts, and the three non-overlapping scripts includes a set of emoji characters, Chinese characters, and Latin script.
13. The non-transitory computer-readable medium of claim 8 , the one or more programs further including instructions for:
identifying a second semantic unit corresponding to the first emoji character recognized from the handwriting input;
displaying, in the candidate display area of the handwriting input interface, a fourth recognition result comprising at least the second semantic unit identified from the first emoji character.
14. The non-transitory computer-readable medium of claim 13 , wherein displaying the fourth recognition result further comprises:
concurrently displaying the fourth recognition result with the first recognition result in the candidate display area.
15. An electronic device, comprising:
a display;
a touch-sensitive surface;
one or more processors; and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
receiving, via the touch-sensitive surface, a handwriting input from a user, the handwriting input comprising a plurality of handwritten strokes provided in a handwriting input area of a handwriting input interface;
recognizing, based on a handwriting recognition model, a plurality of output characters from the handwriting input, the plurality of output characters including at least a first emoji character and at least a first character from a script of a natural human language; and
displaying, on the display, a first recognition result comprising the first emoji character and the first character from the script of the natural human language in a candidate display area of the handwriting input interface.
16. The electronic device of claim 15 , the one or more programs further including instructions for:
based on the handwriting recognition model, recognizing at least a first semantic unit from the handwriting input, wherein the first semantic unit comprises a respective character, word or phrase that is capable of conveying a respective semantic meaning in a respective human language;
identifying a second emoji character associated with the first semantic unit recognized from the handwriting input; and
displaying, in the candidate display area of the handwriting input interface, a second recognition result comprising at least the second emoji character identified from the first semantic unit.
17. The electronic device of claim 16 , wherein displaying the second recognition result further comprises:
concurrently displaying the second recognition result with a third recognition result comprising at least the first semantic unit.
18. The electronic device of claim 15 , the one or more programs further including instructions for:
receiving a user input selecting the first recognition result displayed in the candidate display area; and
in response to the user input, entering text of the selected first recognition result in a text input area of the handwriting input interface, wherein the text includes at least the first emoji character and the first character from the script of the natural human language.
19. The electronic device of claim 15 , wherein the handwriting recognition model has been trained on a multi-script training corpus comprising writing samples corresponding to characters of at least three non-overlapping scripts, and the three non-overlapping scripts includes a set of emoji characters, Chinese characters, and Latin script.
20. The electronic device of claim 15 , the one or more programs further including instructions for:
identifying a second semantic unit corresponding to the first emoji character recognized from the handwriting input;
displaying, in the candidate display area of the handwriting input interface, a fourth recognition result comprising at least the second semantic unit identified from the first emoji character.
21. The electronic device of claim 20 , wherein displaying the fourth recognition result further comprises:
concurrently displaying the fourth recognition result with the first recognition result in the candidate display area.