IP Library Granted Patent US 9,244,906
Granted Patent B2
US 9,244,906 · App. 13/923,464 · Granted Jan 26, 2016

Text entry at electronic communication device

Inventor: Mohammed Houache (San Diego, CA)
Assignee: BLACKBERRY LIMITED
G06F17/276G06F3/0237G10L15/26G10L2015/226
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,244,906
App. No.
13/923,464
Granted
Jan 26, 2016
Kind
B2
Abstract

Typed input is received at a text field from a keyboard, such as a virtual keyboard displayed at an electronic communication device. Text prediction using the typed input can be performed to obtain at least one text-predicted candidate word and an associated confidence value. Speech recognition can be performed on audio input received via a microphone to obtain at least one speech-recognized candidate word and an associated confidence value, which can be adjusted based on the recency of the audio input. A candidate word having a highest confidence value can be selected from the text-predicted and speech-recognized candidate words for display as a suggestion to the user for selection by the user. The suggested candidate word can be displayed on a fret of the virtual keyboard.

Claims (49)

1. A method of facilitating text entry at an electronic communication device, the method comprising:

receiving typed input at a text field from a keyboard of the electronic communication device;

receiving audio input at the electronic communication device;

performing text prediction using the typed input to obtain at least one text-predicted candidate word and an associated confidence value;

performing speech recognition using the audio input to obtain at least one speech-recognized candidate word and an associated confidence value;

selecting from the text-predicted and speech-recognized candidate words a candidate word having a highest associated confidence value, as a suggested candidate word; and, one of:

receiving a portion of the audio input corresponding to a particular speech-recognized candidate word before receiving a most recent portion of the typed input; and, determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received before the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases; or,

receiving the portion of the audio input corresponding to the particular speech-recognized candidate word after receiving the most recent portion of the typed input and, determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received after receiving the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases.

2. The method of claim 1 , further comprising obtaining the speech-recognized candidate word by performing a pre-selection from a plurality of speech-recognized candidate words based on at least a portion of the typed input.

3. The method of claim 2 , wherein the portion of the typed input comprises an initial character of a word.

4. The method of claim 1 , further comprising triggering the text prediction, the selecting, and the entering in response to receiving each character of the typed input.

5. The method of claim 1 , wherein the selecting comprises selecting from the text-predicted and speech-recognized candidate words a plurality of candidate words having highest associated confidence values, as suggested candidate words.

6. The method of claim 5 , further comprising displaying the suggested candidate words as user-selectable on a display of the electronic communication device.

7. The method of claim 6 , further comprising displaying at least two of the suggested candidate words superimposed on a virtual keyboard.

8. The method of claim 6 , further comprising receiving a user selection of one of the suggested candidate words.

9. The method of claim 1 , further comprising entering the suggested candidate word into a text buffer of the electronic communication device.

10. A non-transitory computer-readable medium storing a computer program, wherein execution of the computer program is for: facilitating text entry at an electronic communication device by:

receiving typed input at a text field from a keyboard of the electronic communication device;

receiving audio input at the electronic communication device;

performing text prediction using the typed input to obtain at least one text-predicted candidate word and an associated confidence value;

performing speech recognition using the audio input to obtain at least one speech-recognized candidate word and an associated confidence value;

selecting from the text-predicted and speech-recognized candidate words a candidate word having a highest associated confidence value, as a suggested candidate word; and, one of:

receive a portion of the audio input corresponding to a particular speech-recognized candidate word before receiving a most recent portion of the typed input; and determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received before the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases; or,

receive the portion of the audio input corresponding to the particular speech-recognized candidate word after receiving the most recent portion of the typed input; and, determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received after receiving the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases.

11. The non-transitory computer-readable medium of claim 10 , wherein execution of the computer program is further for: entering the suggested candidate word into a text buffer of the electronic communication device.

12. An electronic communication device comprising:

a display;

a microphone and connected analog-to-digital converter (ADC);

memory; and

a processor connected to the display, the ADC, and the memory, the processor configured to:

output a virtual keyboard to the display;

receive typed input from the keyboard, and perform text prediction using the typed input to obtain at least one text-predicted candidate word and an associated confidence value;

receive audio input from the ADC, and perform speech recognition using the audio input to obtain at least one speech-recognized candidate word and an associated confidence value;

select a candidate word having a highest confidence value from the text-predicted and speech-recognized candidate words, as a suggested candidate word; and, one of:

receiving a portion of the audio input corresponding to a particular speech-recognized candidate word before receiving a most recent portion of the typed input; and, determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received before the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases; or,

receiving the portion of the audio input corresponding to the particular speech-recognized candidate word after receiving the most recent portion of the typed input; and, determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received after receiving the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases.

13. The electronic communication device of claim 12 , wherein the processor is further configured to store the suggested candidate word in the memory.

14. A method of facilitating text entry at an electronic communication device, the method comprising:

receiving typed input from a virtual keyboard;

receiving a stream of digitized audio input;

performing text prediction on a most recent word or word portion of the typed input to obtain at least one text-predicted candidate word and an associated confidence value;

performing speech recognition on the audio input to obtain at least one speech-recognized candidate word and an associated confidence value, the confidence value of the speech-recognized candidate word being based at least in part on a recency of a respective portion of the audio input as compared to a time of receiving the word or word portion of the typed input;

selecting from the text-predicted and speech-recognized candidate words a candidate word, as a suggested candidate word, based on the associated confidence values; and, one of:

receiving a portion of the audio input corresponding to a particular speech-recognized candidate word before receiving a most recent portion of the typed input; and, determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received before the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases; or,

receiving the portion of the audio input corresponding to the particular speech-recognized candidate word after receiving the most recent portion of the typed input; and, determining the confidence value of the particular speech-recognized candidate word by referencing an amount of time that the particular speech-recognized candidate word is received after receiving the most recent portion of the typed input, the confidence value of the particular speech-recognized candidate word decreasing as the amount of time increases.

15. The method of claim 14 , further comprising pre-selecting speech-recognized candidate words based on a match with the most recent word or word portion of the typed input.

16. The method of claim 14 , further comprising displaying the suggested candidate word superimposed on a virtual keyboard.

17. The method of claim 14 , comprising performing the speech recognition as a background process and performing the text prediction and the selecting in response to a key press at the virtual keyboard.

18. The method of claim 14 , further comprising storing the suggested candidate word in a memory of the electronic communication device.

Assignments (5)
NUNC PRO TUNC ASSIGNMENT Recorded Jun 19, 2023
From: BLACKBERRY LIMITED
To: MALIKIE INNOVATIONS LIMITED
Reel/Frame 064271/0199 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 16, 2023
From: BLACKBERRY LIMITED
To: MALIKIE INNOVATIONS LIMITED
Reel/Frame 064104/0103 →
CHANGE OF NAME Recorded Nov 3, 2015
From: RESEARCH IN MOTION LIMITED
To: BLACKBERRY LIMITED
Reel/Frame 037033/0643 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 11, 2013
From: RESEARCH IN MOTION CORPORATION
To: RESEARCH IN MOTION LIMITED
Reel/Frame 030777/0341 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 21, 2013
From: HOUACHE, MOHAMMED
To: RESEARCH IN MOTION CORPORATION
Reel/Frame 030662/0140 →
Continuity (1)
Related Publication 20140379325A1 · Dec 25, 2014