IP Library Granted Patent US 10,741,182
Granted Patent B2
US 10,741,182 · App. 14/182,875 · Granted Aug 11, 2020

Voice input correction using non-audio based input

Inventors: Russell Speight VanBlon (Raleigh, NC); Rod D. Waltermann (Rougemont, NC); Suzanne Marion Beaumont (Wake Forest, NC)
Assignee: Lenovo (Singapore) Pte. Ltd.
G10L15/26G06F40/00G10L15/24G10L15/25
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,741,182
App. No.
14/182,875
Granted
Aug 11, 2020
Kind
B2
Abstract

An embodiment provides a method, including: accepting, at an audio receiver of an information handling device, voice input of a user; interpreting, using a processor, the voice input; identifying, using a processor, at least one ambiguity in interpreting the voice input; thereafter accessing stored non-audible input associated in time with the at least one ambiguity; and adjusting an interpretation of the voice input using non-audible input. Other aspects are described and claimed.

Claims (38)

1. A method, comprising:

accepting, at an audio receiver of an information handling device, voice input of a user and capturing, using a sensor, non-audio based input correlated with the voice input;

generating, using one or more speech recognition engines, an initial interpretation by interpreting the voice input without utilizing the non-audio based input for the initial interpretation;

identifying, using the one or more speech recognition engines, an ambiguous voice input comprising at least one ambiguity in the initial interpretation, wherein the identifying comprises identifying that at least a portion of the initial interpretation is associated with a confidence score meeting a predetermined low confidence threshold, wherein the confidence score is based in part on a condition of the user;

thereafter augmenting the one or more speech recognition engines and re-interpreting the ambiguous voice input by accessing, using the one or more speech recognition engines, based upon the confidence score meeting the predetermined low confidence threshold, stored non-audio based input matched in time with the ambiguous voice input, wherein the accessing is based upon a policy associated with a confidence level of interpretation, wherein the confidence level of interpretation is based on a device usage history, wherein the re-interpreting comprises mapping the stored non-audio based input to known features of the user while providing voice input correlated with the voice input; and

adjusting the initial interpretation of the voice input using non-audio based input, wherein the adjusting comprises changing the initial interpretation using the non-audio based input.

2. The method of claim 1 , wherein the adjusting comprises correcting the voice input.

3. The method of claim 1 , wherein the sensor is a camera.

4. The method of claim 1 , wherein said capturing proceeds according to the policy.

5. The method of claim 1 , wherein a factor comprises at least one factor selected from the group consisting of: history of low speech recognition confidence, detection of multiple speech candidates, a detection of background noise exceeding a predetermined threshold, detection of a repeated word, and detection of an atypical voice characteristic.

6. The method of claim 4 , wherein said policy adjusts said capturing responsive to battery level falling below a predetermined threshold.

7. The method of claim 1 , wherein the accessing stored non-audio based input associated in time with the at least one ambiguity comprises accessing non-audible input derived from data selected from the group consisting of visible light image data, non-visible electromagnetic radiation image data, and non-audible sound data.

8. The method of claim 1 , wherein the identifying at least one ambiguity in interpreting the voice input comprises identifying a word including a predetermined sound characteristic associated with ambiguity.

9. The method of claim 8 , wherein the predetermined sound characteristic associated with ambiguity is a consonant sound.

10. An information handling device, comprising:

an audio receiver;

a sensor that captures input;

one or more processors; and

a memory storing instructions that are executed by processor to:

accept, at the audio receiver, voice input of a user and capture, using the sensor, non-audio based input correlated with the voice input;

generate, using a speech recognition engine, an initial interpretation by interpreting the voice input without utilizing the non-audio based input for the initial interpretation;

identify an ambiguous voice input comprising at least one ambiguity in the initial interpretation, wherein the identifying comprises identifying that at least a portion of the initial interpretation is associated with a confidence score meeting a predetermined low confidence threshold, wherein the confidence score is based in part on a condition of the user;

thereafter augmenting the speech recognition engine and re-interpreting the ambiguous voice input by accessing, using the one or more processors, based upon the confidence score meeting the predetermined low confidence threshold, stored non-audio based input matched in time with the ambiguous voice input, wherein the accessing is based upon a policy associated with a confidence level of interpretation, wherein the confidence level of interpretation is based on a device usage history, wherein the re-interpreting comprises mapping the stored non-audio based input to known features of the user while providing voice input correlated with the voice input; and

adjust the initial interpretation of the voice input using non-audio based input derived from the sensor, wherein to adjust comprises to change the initial interpretation using the non-audio based input.

11. The information handling device of claim 10 , wherein to adjust comprises correcting the voice input.

12. The information handling device of claim 10 , wherein the sensor is a camera.

13. The information handling device of claim 10 , wherein to capture comprises capturing non-audio based input according to the policy.

14. The information handling device of claim 10 , a factor comprises at least one factor selected from the group consisting of: history of low speech recognition confidence, detection of multiple speech candidates, a detection of background noise exceeding a predetermined threshold, detection of a repeated word, and detection of an atypical voice characteristic.

15. The information handling device of claim 13 , wherein said policy adjusts said capturing responsive to battery level falling below a predetermined threshold.

16. The information handling device of claim 10 , wherein to access stored non-audio based input associated in time with the at least one ambiguity comprises accessing non-audio based input derived from data selected from the group consisting of visible light image data, non-visible electromagnetic radiation image data, and non-audible sound image data.

17. The information handling device of claim 10 , wherein the identifying at least one ambiguity in interpreting the voice input comprises identifying a word including a predetermined sound characteristic associated with ambiguity.

18. A product, comprising:

a storage medium having device readable code stored therewith, the device readable code being executable by a processor and comprising:

code that accepts voice input of a user and code that captures non-audio based input correlated with the voice input;

code that generates, using a speech recognition engine, an initial interpretation by interpreting the voice without utilizing the non-audio based input for the initial interpretation;

code that identifies an ambiguous voice input comprising at least one ambiguity in the initial interpretation, wherein the identifying comprises identifying that at least a portion of the initial interpretation is associated with a confidence score meeting a predetermined low confidence threshold, wherein the confidence score is based in part on a condition of the user;

code that thereafter augmenting the speech recognition engine and re-interpreting the ambiguous voice input by accessing, based upon the confidence score meeting the predetermined low confidence threshold, stored non-audio based input matched in time with the ambiguous voice input, wherein the accessing is based upon a policy associated with a confidence level of interpretation, wherein the confidence level of interpretation is based on a device usage history, wherein the re-interpreting comprises mapping the stored non-audio based input to known features of the user while providing voice input correlated with the voice input; and

code that adjusts the initial interpretation of the voice input using non-audio based input, wherein the code that adjusts comprises code that changes the initial interpretation using the non-audio based input.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 10, 2025
From: LENOVO PC INTERNATIONAL LIMITED
To: LENOVO SWITZERLAND INTERNATIONAL GMBH
Reel/Frame 069870/0670 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 14, 2022
From: LENOVO (SINGAPORE) PTE LTD
To: LENOVO PC INTERNATIONAL LIMITED
Reel/Frame 060651/0634 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 18, 2014
From: VANBLON, RUSSELL SPEIGHT; WALTERMANN, ROD D.; BEAUMONT, SUZANNE MARION
To: LENOVO (SINGAPORE) PTE. LTD.
Reel/Frame 032236/0340 →