IP Library Granted Patent US 12688455
Granted Patent B2
US 12688455 · App. 17/517,374 · Granted Jul 21, 2026

Detecting human input activity in a cognitive environment using wearable inertia and audio sensors

Inventor: Victor C. Dibia (Whiteplains, NY)
Assignee: International Business Machines Corporation
G06N20/00A61B5/1118A61B5/681A61B5/6824A61B5/165G06F1/163G06N5/02
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12688455
App. No.
17/517,374
Filed
Nov 2, 2021
Granted
Jul 21, 2026
Kind
B2
Art Unit
2124
USPC
706/46
Abstract

A mechanism is provided in a data processing system comprising a processor and a memory. The memory comprises instructions which are executed by the processor to cause the processor to be specifically configured to implement a recognizer module for detecting user input in a cognitive environment. The recognizer module receives sensor signals from at least one wearable device being worn by a user. The recognizer module analyzes the sensor signals using a machine learning model to determine at least one user input indicator describing user input activity of the user. The recognizer module communicates the at least one user input indicator to a cognitive system executing within the cognitive environment. The cognitive system performs at least one cognitive action based on the at least one user input indicator.

Claims (37)

1 . A method, in a data processing system comprising a processor and a memory wherein the memory comprises instructions which are executed by the processor, the method comprising:

receiving, by a recognizer module executing within the data processing system, sensor signals from at least one wearable device being worn by a user;

analyzing, by the recognizer module, the sensor signals using a machine learning model to determine at least one user input indicator describing user input activity of the user, wherein the at least one user input indicator comprises a wrist angle indicator that indicates an estimated wrist angle associated with the user input activity of the user;

identifying that a change in the estimated wrist angle relative to a baseline wrist position is associated with a health condition;

communicating, by the recognizer module, the at least one user input indicator to a cognitive system executing within the data processing system; and

activating at least one actuator based on identifying that the change in the estimated wrist angle is associated with the health condition.

2 . The method of claim 1 , wherein the sensor signals comprise at least one vibration signal from an accelerometer or gyroscope in the at least one wearable device.

3 . The method of claim 1 , wherein the at least one actuator comprises an actuator to adjust a chair height.

4 . The method of claim 1 , wherein the at least one actuator comprises an actuator to adjust a keyboard configuration.

5 . The method of claim 1 , wherein the at least one user input indicator comprises a stress level indicator that indicates stress level associated with the user input activity of the user.

6 . The method of claim 5 , wherein activating the at least one actuator comprises activating at least one of an actuator to dim lighting, an ambient lighting actuator, an actuator to control music, an actuator to control temperature.

7 . The method of claim 1 , wherein the machine learning model is a user-specific machine learning model trained for the user of the at least one wearable device.

8 . The method of claim 1 , wherein the at least one wearable device includes a smartwatch device or a fitness band.

9 . A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on one or more processors of a data processing system, causes the data processing system to:

receive, by a recognizer module executing within the data processing system, sensor signals from at least one wearable device being worn by a user;

analyze, by the recognizer module, the sensor signals using a machine learning model to determine at least one user input indicator describing user input activity of the user, wherein the at least one user input indicator comprises a wrist angle indicator that indicates an estimated wrist angle associated with the user input activity of the user;

identify that a change in the estimated wrist angle relative to a baseline wrist position is associated with a health condition;

communicate, by the recognizer module, the at least one user input indicator to a cognitive system executing within the data processing system; and

actuate at least one actuator based on identification that the change in the estimated wrist angle is associated with the health condition.

10 . The computer program product of claim 9 , wherein the sensor signals comprise at least one vibration signal from an accelerometer or gyroscope in the at least one wearable device.

11 . The computer program product of claim 9 , wherein the at least one actuator comprises an actuator to adjust a chair height.

12 . The computer program product of claim 9 , wherein the at least one user input indicator comprises a stress level indicator that indicates stress level associated with the user input activity of the user.

13 . The computer program product of claim 12 , wherein activating the at least one actuator comprises activating at least one of an actuator to dim lighting, an ambient lighting actuator, an actuator to control music, an actuator to control temperature.

14 . The computer program product of claim 9 , wherein the at least one wearable device includes a smartwatch device or a fitness band.

15 . An apparatus within a data processing system comprising:

one or more processors; and

one or more memories coupled to the processor, wherein the one or more memories comprises instructions which, when executed by the one or more processors, cause the one or more processors to:

receive, by a recognizer module executing within the data processing system, sensor signals from at least one wearable device being worn by a user;

analyze, by the recognizer module, the sensor signals using a machine learning model to determine at least one user input indicator describing user input activity of the user, wherein the at least one user input indicator comprises a wrist angle indicator that indicates an estimated wrist angle associated with the user input activity of the user;

identify that a change in the estimated wrist angle relative to a baseline wrist position is associated with a health condition;

communicate, by the recognizer module, the at least one user input indicator to a cognitive system executing within the data processing system; and

activate at least one actuator based on identification that the change in the estimated wrist angle is associated with the health condition.

16 . The computer program product of claim 9 , wherein the at least one actuator comprises an actuator to adjust a keyboard configuration.

17 . The computer program product of claim 9 , wherein the machine learning model is a user-specific machine learning model trained for the user of the at least one wearable device.

18 . The apparatus of claim 15 , wherein the at least one actuator comprises an actuator to adjust a chair height.

19 . The apparatus of claim 15 , wherein the at least one actuator comprises an actuator to adjust a keyboard configuration.

20 . The apparatus of claim 15 , wherein the machine learning model is a user-specific machine learning model trained for the user of the at least one wearable device.