IP Library › Granted Patent US 11,335,342
Granted Patent B2
US 11,335,342 · App. 16/797,604 · Granted May 17, 2022

Voice assistance system

Inventors: Raghuveer Prasad Nagar (Kota, IN); Sarbajit K. Rakshit (Kolkata, IN); Jagadesh Ramaswamy Hulugundi (Bangalore, IN); Reji Jose (Bangalore, IN)
Assignee: International Business Machines Corporation
G10L15/22A61B3/113A61B5/165G02B27/0093G02B27/0101G02C7/04G06F3/167G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,335,342
App. No.
16/797,604
Granted
May 17, 2022
Kind
B2
Abstract

An approach is provided in which the approach captures a voice command spoken by a user along with a set of data generated from a smart contact lens worn by the user. The approach matches the set of data to a command augmentation indicator that identifies an augmentation to the voice command. The approach aggregates the command augmentation indicator with the voice command into an aggregated command and executes the aggregated command accordingly.

Claims (68)

1. A method implemented by an information handling system that includes a memory and a processor, the method comprising:

capturing a voice command and a set of oculesics data corresponding to a user, wherein the voice command is spoken by the user and the set of oculesics data is generated from a smart contact lens worn by the user;

matching the set of oculesics data to a command augmentation indicator, wherein the command augmentation indicator indicates an augmentation to the voice command;

aggregating the command augmentation indicator with the voice command into an aggregated command;

executing the aggregated command by a voice assistance system;

receiving a different set of oculesics data from the smart contact lens;

determining that the different set of oculesics data corresponds to a runtime control operation; and

executing the runtime control operation by the voice assistance system.

2. The method of claim 1 wherein the set of oculesics data is selected from a group consisting of eye lid data, eye tear data, and eye pupil data.

3. The method of claim 2 further comprising:

capturing the different set of oculesics data in response to executing the aggregated command;

determining an emotional state of the user based on the different set of oculesics data; and

adjusting a future aggregated command based on the determined emotional state.

4. The method of claim 2 further comprising:

receiving the set of oculesics data and the voice command at a mobile device utilized by the user;

performing, by the mobile device, the matching of the set of oculesics data to the command augmentation indicator and the aggregating of the command augmentation indicator with the voice command; and

transmitting the aggregated command from the mobile device to the voice assistance system.

5. The method of claim 2 further comprising:

waking up the voice assistance system based on the command augmentation indicator.

6. The method of claim 1 further comprising:

inhibiting the executing of the aggregated command in response to determining that the command augmentation indicator indicates not to execute the voice command.

7. The method of claim 1 further comprising:

prior to the capturing of the set of oculesics data:

capturing a set of oculesics training data from the smart contact lens;

receiving the command augmentation indicator from the user;

creating a table entry that combines the set of oculesics training data with the command augmentation indicator; and

storing the table entry in a data translation table; and

accessing the data translation table during the matching of the set of oculesics data to the command augmentation indicator.

8. The method of claim 1 wherein the smart contact lens comprises a tears sensor, an eyelid sensor, and a data transmitter.

9. An information handling system comprising:

one or more processors;

a memory coupled to at least one of the processors;

a set of computer program instructions stored in the memory and executed by at least one of the processors in order to perform actions of:

capturing a voice command and a set of oculesics data corresponding to a user, wherein the voice command is spoken by the user and the set of oculesics data is generated from a smart contact lens worn by the user;

matching the set of oculesics data to a command augmentation indicator, wherein the command augmentation indicator indicates an augmentation to the voice command;

aggregating the command augmentation indicator with the voice command into an aggregated command;

executing the aggregated command by a voice assistance system;

receiving a different set of oculesics data from the smart contact lens;

determining that the different set of oculesics data corresponds to a runtime control operation; and

executing the runtime control operation by the voice assistance system.

10. The information handling system of claim 9 wherein the set of oculesics data is selected from a group consisting of eye lid data, eye tear data, and eye pupil data.

11. The information handling system of claim 10 wherein the processors perform additional actions comprising:

capturing the different set of oculesics data in response to executing the aggregated command;

determining an emotional state of the user based on the different set of oculesics data; and

adjusting a future aggregated command based on the determined emotional state.

12. The information handling system of claim 10 wherein the processors perform additional actions comprising:

receiving the set of oculesics data and the voice command at a mobile device utilized by the user;

performing, by the mobile device, the matching of the set of oculesics data to the command augmentation indicator and the aggregating of the command augmentation indicator with the voice command; and

transmitting the aggregated command from the mobile device to the voice assistance system.

13. The information handling system of claim 9 wherein the processors perform additional actions comprising:

inhibiting the executing of the aggregated command in response to determining that the command augmentation indicator indicates not to execute the voice command.

14. A computer program product stored in a computer readable storage medium, comprising computer program code that, when executed by an information handling system, causes the information handling system to perform actions comprising:

capturing a voice command and a set of oculesics data corresponding to a user, wherein the voice command is spoken by the user and the set of oculesics data is generated from a smart contact lens worn by the user;

matching the set of oculesics data to a command augmentation indicator, wherein the command augmentation indicator indicates an augmentation to the voice command;

aggregating the command augmentation indicator with the voice command into an aggregated command;

executing the aggregated command by a voice assistance system;

receiving a different set of oculesics data from the smart contact lens;

determining that the different set of oculesics data corresponds to a runtime control operation; and

executing the runtime control operation by the voice assistance system.

15. The computer program product of claim 14 wherein the set of oculesics data is selected from a group consisting of eye lid data, eye tear data, and eye pupil data.

16. The computer program product of claim 15 wherein the information handling system performs further actions comprising:

capturing the different set of oculesics data in response to executing the aggregated command;

determining an emotional state of the user based on the different set of oculesics data; and

adjusting a future aggregated command based on the determined emotional state.

17. The computer program product of claim 15 wherein the information handling system performs further actions comprising:

receiving the set of oculesics data and the voice command at a mobile device utilized by the user;

performing, by the mobile device, the matching of the set of oculesics data to the command augmentation indicator and the aggregating of the command augmentation indicator with the voice command; and

transmitting the aggregated command from the mobile device to the voice assistance system.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 21, 2020
From: NAGAR, RAGHUVEER PRASAD; RAKSHIT, SARBAJIT K.; HULUGUNDI, JAGADESH RAMASWAMY; JOSE, REJI
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 051888/0967 →
Continuity (1)
Related Publication 20210264908A1 · Aug 26, 2021