Managing real-time handwriting recognition
Methods, systems, and computer-readable media related to a technique for providing handwriting input functionality on a user device. A handwriting recognition module is trained to have a repertoire comprising multiple non-overlapping scripts and capable of recognizing tens of thousands of characters using a single handwriting recognition model. The handwriting input module provides real-time, stroke-order and stroke-direction independent handwriting recognition for multi-character handwriting input. In particular, real-time, stroke-order and stroke-direction independent handwriting recognition is provided for multi-character, or sentence level Chinese handwriting recognition. User interfaces for providing the handwriting input functionality are also disclosed.
1. A non-transitory computer-readable media comprising instructions stored thereon, the instructions, when executed by one or more processors, cause the processors to perform operations comprising:
receiving a handwriting input from a user, the handwriting input comprising a plurality of handwritten strokes provided in a touch-sensitive surface coupled to the device;
rendering, in real-time, the plurality of handwritten strokes in a handwriting input area of a handwriting input interface;
receiving one of a pinch gesture input and an expand gesture input over the plurality of handwritten strokes;
upon identifying two adjacent recognition units from the plurality of handwritten strokes and upon the one of a pinch gesture input and an expand gesture input being a pinch gesture input:
prior to receiving the pinch gesture input, displaying, in a candidate display area, a first initial recognition result comprising respective initial output characters recognized from the two adjacent recognition units;
subsequent to receiving the pinch gesture input, generating a first subsequent recognition result based on the plurality of handwritten strokes by treating the plurality of handwritten strokes as a single recognition unit;
replacing the first initial recognition result with the first subsequent recognition result in the candidate display area; and
in response to the pinch gesture input, re-rendering the plurality of handwritten strokes to reduce a distance between portions of the handwritten strokes corresponding to the two adjacent recognition units in the handwriting input area;
upon identifying a singular recognition unit from the plurality of handwritten strokes and upon the one of a pinch gesture input and an expand gesture input being an expand gesture input:
prior to receiving the expand gesture input, displaying, in the candidate display area, a second initial recognition result comprising an initial output character recognized from the singular recognition unit;
subsequent to receiving the expand gesture input, generating a second subsequent recognition result based on the plurality of handwritten strokes by treating the plurality of handwritten strokes as two separate recognition units pulled apart by the expand gesture input;
replacing the second initial recognition result with the second recognition result in the candidate display area, and
in response to the expand gesture input, re-rendering the plurality of handwritten strokes to increase a distance between portions of the handwritten strokes corresponding to the singular recognition unit in the handwriting input area.
2. The media of claim 1 , wherein the pinch gesture input comprises two contacts on the touch-sensitive surface that converge toward each other in an area occupied by the plurality of handwritten strokes.
3. The media of claim 1 , wherein the expand gesture input comprises two contacts on the touch-sensitive surface that diverge from each other in an area occupied by the plurality of handwritten strokes.
4. A method of providing handwriting recognition, comprising: at a device having one or more processors and memory:
receiving a handwriting input from a user, the handwriting input comprising a plurality of handwritten strokes provided in a touch-sensitive surface coupled to the device;
rendering, in real-time, the plurality of handwritten strokes in a handwriting input area of a handwriting input interface;
receiving one of a pinch gesture input and an expand gesture input over the plurality of handwritten strokes;
upon identifying two adjacent recognition units from the plurality of handwritten strokes and upon the one of a pinch gesture input and an expand gesture input being a pinch gesture input:
prior to receiving the pinch gesture input, displaying, in a candidate display area, a first initial recognition result comprising respective initial output characters recognized from the two adjacent recognition units;
subsequent to receiving the pinch gesture input, generating a first subsequent recognition result based on the plurality of handwritten strokes by treating the plurality of handwritten strokes as a single recognition unit;
replacing the first initial recognition result with the first subsequent recognition result in the candidate display area; and
in response to the pinch gesture input, re-rendering the plurality of handwritten strokes to reduce a distance between portions of the handwritten strokes corresponding to the two adjacent recognition units in the handwriting input area;
upon identifying a singular recognition unit from the plurality of handwritten strokes and upon the one of a pinch gesture input and an expand gesture input being an expand gesture input:
prior to receiving the expand gesture input, displaying, in the candidate display area, a second initial recognition result comprising an initial output character recognized from the singular recognition unit;
subsequent to receiving the expand gesture input, generating a second subsequent recognition result based on the plurality of handwritten strokes by treating the plurality of handwritten strokes as two separate recognition units pulled apart by the expand gesture input;
replacing the second initial recognition result with the second recognition result in the candidate display area, and
in response to the expand gesture input, re-rendering the plurality of handwritten strokes to increase a distance between portions of the handwritten strokes corresponding to the singular recognition unit in the handwriting input area.
5. A system, comprising:
one or more processors; and
memory having instructions stored thereon, the instructions, when executed by the one or more processors, cause the processors to perform operations comprising:
receiving a handwriting input from a user, the handwriting input comprising a plurality of handwritten strokes provided in a touch-sensitive surface coupled to the device;
rendering, in real-time, the plurality of handwritten strokes in a handwriting input area of a handwriting input interface;
receiving one of a pinch gesture input and an expand gesture input over the plurality of handwritten strokes;
upon identifying two adjacent recognition units from the plurality of handwritten strokes and upon the one of a pinch gesture input and an expand gesture input being a pinch gesture input:
prior to receiving the pinch gesture input, displaying, in a candidate display area, a first initial recognition result comprising respective initial output characters recognized from the two adjacent recognition units;
subsequent to receiving the pinch gesture input, generating a first subsequent recognition result based on the plurality of handwritten strokes by treating the plurality of handwritten strokes as a single recognition unit;
replacing the first initial recognition result with the first subsequent recognition result in the candidate display area; and
in response to the pinch gesture input, re-rendering the plurality of handwritten strokes to reduce a distance between portions of the handwritten strokes corresponding to the two adjacent recognition units in the handwriting input area;
upon identifying a singular recognition unit from the plurality of handwritten strokes and upon the one of a pinch gesture input and an expand gesture input being an expand gesture input:
prior to receiving the expand gesture input, displaying, in the candidate display area, a second initial recognition result comprising an initial output character recognized from the singular recognition unit;
subsequent to receiving the expand gesture input, generating a second subsequent recognition result based on the plurality of handwritten strokes by treating the plurality of handwritten strokes as two separate recognition units pulled apart by the expand gesture input;
replacing the second initial recognition result with the second recognition result in the candidate display area, and
in response to the expand gesture input, re-rendering the plurality of handwritten strokes to increase a distance between portions of the handwritten strokes corresponding to the singular recognition unit in the handwriting input area.
6. The system of claim 5 , wherein the pinch gesture input comprises two contacts on the touch-sensitive surface that converge toward each other in an area occupied by the plurality of handwritten strokes.
7. The system of claim 5 , wherein the expand gesture input comprises two contacts on the touch-sensitive surface that diverge from each other in an area occupied by the plurality of handwritten strokes.
8. The method of claim 4 , wherein the pinch gesture input comprises two contacts on the touch-sensitive surface that converge toward each other in an area occupied by the plurality of handwritten strokes.
9. The method of claim 4 , wherein the expand gesture input comprises two contacts on the touch-sensitive surface that diverge from each other in an area occupied by the plurality of handwritten strokes.