IP Library › Granted Patent US 12,626,525
Granted Patent B2
US 12,626,525 · App. 18/250,312 · Granted May 12, 2026

System for recognizing online handwriting

Inventors: Eric Humbert (Boulogne Billancourt, FR); Arthur Belhomme (Paris, FR); Amélie Caudron (Paris, FR)
Assignee: SOCIÉTÉ BIC
G06V30/2268G06V30/19147G06V30/1918G06V30/228G06V30/347G06V30/36
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,626,525
App. No.
18/250,312
Granted
May 12, 2026
Kind
B2
Abstract

This invention concerns a system for recognizing online handwriting comprising: a handwriting instrument including a body extending longitudinally between a first end and a second end, the first end having a writing tip which is able to write on a support, the handwriting instrument further including a module comprising at least one motion sensor configured to acquire motion data on the handwriting of a user when the user is writing a characters sequence with the handwriting instrument, a calculating unit communicating with the at least one motion sensor and configured to analyze the motion data by a machine learning model trained in a multitask way such that it is capable of performing at least two tasks at the same time, the machine learning model being configured to deliver as an output the characters sequence which was written by the user with the handwriting instrument.

Claims (23)

1 . A system for recognizing an online handwriting of a user comprising:

a handwriting instrument including a body extending longitudinally between a first end and a second end, the first end having a writing tip which is able to write on a support, a module comprising at least one motion sensor configured to acquire motion data on the online handwriting of the user when the user is writing a characters sequence with the handwriting instrument,

a calculating unit comprising a memory and a processor, the calculating unit communicating with the at least one motion sensor and configured to analyze the motion data by a machine learning model trained in a multitask way such that it is capable of performing a stroke segmentation task and a character classification task at a same time, wherein:

the machine learning model comprises a shared backbone neural network configured to perform hidden features extraction of the motion data to obtain intermediate features,

the stroke segmentation task is performed on the intermediate features, and

the character classification task is performed using a recursive neural network using the intermediate features, and

the machine learning model being configured to deliver as an output the characters sequence which was written by the user with the handwriting instrument.

2 . The system according to claim 1 , wherein the module is embedded in the second end of the handwriting instrument.

3 . The system according to claim 1 , wherein the module is placed on an outside surface of the second end of the handwriting instrument.

4 . The system according to claim 1 , wherein the module further comprises the calculating unit.

5 . The system according to claim 1 , wherein the module further includes a short-range radio communication interface configured to communicate raw motion data acquired by the at least one motion sensor to a mobile device comprising the calculating unit via a communication interface of the mobile device.

6 . The system according to claim 1 , wherein the at least one motion sensor is a three-axis accelerometer.

7 . The system according to claim 6 , wherein the module further comprises a second motion sensor being a three-axis gyroscope.

8 . The system according to claim 1 , wherein characters of the characters sequence comprise numbers.

9 . The system according to claim 1 , wherein the calculating unit comprises a volatile memory to store the motion data acquired by the at least one motion sensor.

10 . The system according to claim 1 , wherein the calculating unit comprises a non-volatile memory to store the machine learning model enabling the recognizing of the online handwriting of the user.

11 . The system according to claim 1 , wherein the trained machine learning model is stored in the calculating unit.

12 . The system according to claim 1 , wherein the stroke segmentation task comprises labeling the acquired motion data in at least one of following classes: on-paper stroke or in-air movement.

13 . The system according to claim 12 , wherein the trained machine learning model is further configured such that the acquired motion data is pre-processed before being used in the stroke segmentation task, wherein pre-processing comprises windowing raw motion data in time frames of N samples, wherein the samples are labeled in one of the on-paper stroke or the in-air movement.

14 . The system according to claim 12 , wherein an in-air movement label comprises at least two sub-labels, forward in-air movement and backward in-air movement.

15 . The system according to claim 1 , wherein the module further includes a battery configured to provide power to the at least one motion sensor when the user is using the handwriting instrument.

16 . The system according to claim 15 , wherein the battery is further configured to provide power to other components comprised in the module.

17 . The system according to claim 1 , wherein characters of the characters sequence comprise letters.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 12, 2023
From: HUMBERT, ERIC; BELHOMME, ARTHUR; CAUDRON, AMÉLIE
To: SOCIÉTÉ BIC
Reel/Frame 064222/0640 →
Priority Claims (1)
EP 20306281 · Oct 26, 2020 · regional
Continuity (1)
Related Publication 20230401878A1 · Dec 14, 2023
References Cited (26)
US 10126825B2 · Lefebvre · 2018 [cited by applicant]
US 20160154485A1 · Kampt · 2016 [cited by examiner]
US 20160259436A1 · Nicholson · 2016 [cited by examiner]
US 20160378195A1 · Lefebvre · 2016 [cited by applicant]
US 20180299976A1 · Chiewcharnpipat · 2018 [cited by examiner]
US 20200250413A1 · Lu · 2020 [cited by examiner]
CN 110134257A · 2019 [cited by examiner]
CN 115769177A · 2023 [cited by applicant]
WO WO03025829A2 · 2003 [cited by examiner]
Hsu, Y-L. et al., “An Inertial Pen With Dynamic Time Warping Recognizer for Handwriting and Gesture Recognition,” IEEE Sensors Journal, 15(1), 2015, p. 154-163. (Year: 2015). [cited by examiner]
He, S. & Schomaker, L., “Deep Adaptive Learning for Writer Identification based on Single Handwritten Word Images,” arXiv, 2018, p. 1-24. (Year: 2018). [cited by examiner]
Xie, Z. et al., “Learning Spatial-semantic Context with Fully Convolutional Recurrent Network for Online Handwritten Chinese Text Recognition,” arXiv, 2017, p. 1-14. (Year: 2017). [cited by examiner]
Bashir, M. & Kempf, J., “Reduced Dynamic Time Warping for Handwriting Recognition Based on Multi-dimensional Time Series of a Novel Pen Device,” International Scholarly and Scientific Research & Innovation, 2(9), 2008, … [cited by examiner]
Agrawal, S. et al., “Using Mobile Phones to Write in Air,” MobiSys, 2011, p. 1-14. (Year: 2011). [cited by examiner]
Agrawal et al., “Using Mobile Phones to Write in Air,” MobiSys, 2011, p. 1-14. (Year: 2011). [cited by applicant]
Bashir et al., “Reduced Dynamic Time Warping for Handwriting Recognition Based on Multi-dimensional Time Series of a Novel Pen Device,” International Scholarly and Scientific Research & Innovation, 2(9), 2008, p. 1839-1… [cited by applicant]
Crawshaw, Michael. “Multi-Task Learning with Deep Neural Networks: A Survey.” ArXiv abs/2009.09796 (2020): n. pag. [cited by applicant]
Sheng He and Lambert Schomaker. 2019. “Deep adaptive learning for writer identification based on single handwritten word images.” Pattern Recognition. 88, C (Apr. 2019), 64-74. https://doi.org/10.1016/j.patcog.2018.11.0… [cited by applicant]
Z. Chen, Y. Wu, F. Yin and C.-L. Liu, “Simultaneous Script Identification and Handwriting Recognition via Multi-Task Learning of Recurrent Neural Networks,” 2017 14th IAPR International Conference on Document Analysis a… [cited by applicant]
Hsu Yu-Liang et al: “An Inertial Pen With Dynamic Time Warping Recognizer for Handwriting and Gesture Recognition” IEEE Sensors Journal, IEEE Service Center, New York, NY, US, vol. 15, No. 1, Jan. 1, 2015 (Jan. 1, 2015)… [cited by applicant]
International Search Report issued in International Application No. PCT/EP2021/079431 on Feb. 14, 2022 (4 pages). [cited by applicant]
Tappert CC et al: “The State of the Art in On-Line Handwriting Recognition”, IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Computer Society, USA, vol. 12, No. 8, Aug. 1, 1990 (Aug. 1, 1990), pp. 7… [cited by applicant]
Tasdemir Esma F et al: “A comparative study of delayed stroke handling approaches in online handwriting”, International Journal on Document Analysis and Recognition, Springer, Heidelberg, DE, vol. 22, No. 1, Nov. 13, 20… [cited by applicant]
Wang Chengcheng et al: “Combined Segmentation and Recognition of Online Handwritten Diagrams with High Order Markov Random Field”, 2014 14th International Conference on Frontiers in Handwriting Recognition, IEEE, Oct. 2… [cited by applicant]
Xie, Zecheng & Sun, Zenghui & Jin, Lianwen & Ni, Hao & Lyons, Terry. (2016). Learning Spatial-Semantic Context with Fully Convolutional Recurrent Network for Online Handwritten Chinese Text Recognition. IEEE Transaction… [cited by applicant]
Liu, B. et al. “Offline Handwritten Chinese Text Recognition with Convolutional Neural Networks.” A PyTorch binding at https://github.com/baidu-research/warp-ctc. (2020). [cited by applicant]