IP Library Granted Patent US 10,878,231
Granted Patent B2
US 10,878,231 · App. 15/975,897 · Granted Dec 29, 2020

Writing recognition using wearable pressure sensing device

Inventors: Katsuyuki Sakuma (Fishkill, NY); Stephen J. Heisig (Tarrytown, NY); John J. Rice (Mohegan Lake, NY); John Knickerbocker (Yorktown Heights, NY); Gaddi Blumrosen (New York, NY)
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
G06K9/00409G06F3/014G06F3/0346G06F3/0354G06F3/04883G06K9/209G06K9/40G06K9/42G06K9/72G06F2203/0331G06K2209/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,878,231
App. No.
15/975,897
Granted
Dec 29, 2020
Kind
B2
Abstract

Writing recognition using a wearable pressure sensing device includes receiving pressure measurement data from a pressure sensor disposed upon a body part of a user. The pressure measurement data is indicative of a change in pressure of the body part due to an interaction of the body part with a medium indicative of a writing gesture by the user. A start boundary and end boundary for each of a plurality of writing symbols is detected based upon the pressure measurement data. At least one feature of the pressure measurement data associated with the plurality of writing symbols is extracted. A symbol pattern is detected based upon the extracted features, and at least one letter is detected based upon the symbol pattern. A word is detected based upon the detected at least one letter.

Claims (48)

1. A method of writing recognition using a wearable pressure sensing device comprising:

receiving pressure measurement data from a pressure sensor disposed upon a body part of a user, the pressure measurement data indicative of a change in pressure of the body part due to an interaction of the body part with a medium indicative of a writing gesture by the user;

detecting a start boundary and end boundary for each of a plurality of writing symbols based upon the pressure measurement data, by detecting, as an off state, a reference state wherein the body part is in a steady state, and detecting an on state based on a deviation from the off state, wherein detection of the deviation from the off state is performed with edge detection techniques,

extracting at least one feature of the pressure measurement data associated with the plurality of writing symbols;

detecting a symbol pattern based upon the extracted features;

detecting at least one letter based upon the symbol pattern; and

detecting a word based upon the detected at least one letter.

2. The method of claim 1 , further comprising:

receiving kinetic data from a kinetic sensor disposed upon the body part of a user, wherein the detecting of the symbol pattern is further based upon the kinetic data.

3. The method of claim 1 , further comprising:

reducing a dimensionality of the pressure measurement data prior to the detecting of the start boundary and end boundary.

4. The method of claim 3 , further comprising:

pre-processing the pressure measurement data prior to the reducing of the dimensionality of the pressure measurement data.

5. The method of claim 4 , wherein pre-processing of the pressure measurement data includes one or more of filtering out a bias and undesired frequencies content from the pressure measurement data, scaling the pressure measurement data and removing temporal artifacts from the pressure measurement data.

6. The method of claim 1 , further comprising:

detecting a command based upon the extracted features.

7. The method of claim 1 , further comprising:

detecting a symbol of a virtual keyboard based upon the extracted features.

8. The method of claim 1 , further comprising:

detecting a medical condition of the user based upon the extracted features.

9. The method of claim 1 , wherein the at least one feature includes one or more of a spatial feature, a temporal feature, and a spectral feature of the pressure measurement data.

10. The method of claim 1 , wherein detecting of the symbol pattern is based upon prior statistical knowledge.

11. The method of claim 1 , wherein the detecting of the at least one letter is performed using at least one of machine learning on the extracted features or on raw data of the pressure sensor with additional sensor data.

12. A computer usable program product comprising one or more computer-readable storage devices, and program instructions stored on at least one of the one or more storage devices, the stored program instructions comprising:

program instructions to receive pressure measurement data from a pressure sensor disposed upon a body part of a user, the pressure measurement data indicative of a change in pressure of the body part due to an interaction of the body part with a medium indicative of a writing gesture by the user;

program instructions to detect a start boundary and end boundary for each of a plurality of writing symbols based upon the pressure measurement data, by detecting, as an off state, a reference state wherein the body part is in a steady state, and detecting an on state based on a deviation from the off state, wherein detection of the deviation from the off state is performed with edge detection techniques,

program instructions to extract at least one feature of the pressure measurement data associated with the plurality of writing symbols;

program instructions to detect a symbol pattern based upon the extracted features;

program instructions to detect at least one letter based upon the symbol pattern; and

program instructions to detect a word based upon the detected at least one letter.

13. The computer usable program product of claim 12 , further comprising:

program instructions to receive kinetic data from a kinetic sensor disposed upon the body part of a user, wherein the detecting of the symbol pattern is further based upon the kinetic data.

14. The computer usable program product of claim 12 , further comprising:

program instructions to reduce a dimensionality of the pressure measurement data prior to the detecting of the start boundary and end boundary.

15. The computer usable program product of claim 14 , further comprising:

program instructions to pre-process the pressure measurement data prior to the reducing of the dimensionality of the pressure measurement data.

16. The computer usable program product of claim 15 , wherein pre-processing of the pressure measurement data includes one or more of filtering out a bias and undesired frequencies content from the pressure measurement data, scaling the pressure measurement data and removing temporal artifacts from the pressure measurement data.

17. The computer usable program product of claim 12 , further comprising: further comprising:

program instructions to detect a command based upon the extracted features.

18. The computer usable program product of claim 12 , wherein the computer usable code is stored in a computer readable storage device in a data processing system, and wherein the computer usable code is transferred over a network from a data processing system.

19. The computer usable program product of claim 12 , wherein the computer usable code is stored in a computer readable storage device in a server data processing system, and wherein the computer usable code is downloaded over a network to a data processing system for use in a computer readable storage device associated with the data processing system.

20. A computer system comprising one or more processors, one or more computer-readable memories, and one or more computer-readable storage devices, and program instructions stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, the stored program instructions comprising:

program instructions to receive pressure measurement data from a pressure sensor disposed upon a body part of a user, the pressure measurement data indicative of a change in pressure of the body part due to an interaction of the body part with a medium indicative of a writing gesture by the user, by detecting, as an off state, a reference state wherein the body part is in a steady state, and detecting an on state based on a deviation from the off state, wherein detection of the deviation from the off state is performed with edge detection techniques,

program instructions to detect a start boundary and end boundary for each of a plurality of writing symbols based upon the pressure measurement data;

program instructions to extract at least one feature of the pressure measurement data associated with the plurality of writing symbols;

program instructions to detect a symbol pattern based upon the extracted features;

program instructions to detect at least one letter based upon the symbol pattern; and

program instructions to detect a word based upon the detected at least one letter.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 30, 2018
From: RICE, JOHN J
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 045926/0854 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 10, 2018
From: SAKUMA, KATSUYUKI; HEISIG, STEPHEN J.; KNICKERBOCKER, JOHN; BLUMROSEN, GADDI
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 045764/0531 →
Continuity (1)
Related Publication 20190347479A1 · Nov 14, 2019