CONTROLLING ELECTRONIC COMPONENTS IN A COMPUTING ENVIRONMENT
The present invention is directed toward a system and process that controls a group of networked electronic components using a multimodal integration scheme in which inputs from a speech recognition subsystem, gesture recognition subsystem employing a wireless pointing device and pointing analysis subsystem also employing the pointing device, are combined to determine what component a user wants to control and what control action is desired. In this multimodal integration scheme, the desired action concerning an electronic component is decomposed into a command and a referent pair. The referent can be identified using the pointing device to identify the component by pointing at the component or an object associated with it, by using speech recognition, or both. The command may be specified by pressing a button on the pointing device, by a gesture performed with the pointing device, by a speech recognition event, or by any combination of these inputs.
1 . A movement recognition apparatus comprising:
receiving means for receiving a signal via a wireless medium, the signal originating from a pointing device;
demodulation means for demodulating the received signal;
control means for receiving the demodulated signal and providing a plurality of output data messages, the plurality of output data messages including at least one acceleration data value detected by an accelerometer during a motion of the pointing device;
storage means for storing the plurality of output data messages as well as a plurality of threshold definitions associated with accelerometer output of the pointing device, wherein each of the plurality of threshold definitions includes at least one threshold value utilized to indicate that the motion of the pointing device has occurred; and
processing means for processing the plurality of output data messages, wherein the processing includes accessing each of the plurality of threshold definitions, comparing each of the threshold definitions to the plurality of output data messages, identifying whether the at least one threshold value in each of the threshold definitions is exceeded by any of the plurality of output data messages, and selecting one of the threshold definitions, wherein the at least one threshold value in the selected one of the threshold definitions is exceeded by any of the plurality of output data messages.
2 . One or more computer-readable storage media in a computing device having computer-useable instructions embodied thereon for performing a method for recognizing a movement associated with a pointing device, the method comprising:
referencing a threshold associated with the pointing device;
receiving sensor data at the computing device, the sensor data being output from the pointing device, the sensor data representing a motion of the pointing device;
identifying whether the threshold is exceeded by the sensor data; and
determining that the movement has occurred, if the threshold value is exceeded by the sensor data.
3 . The one or more computer-readable storage media of claim 2 further comprising initiating a control action by the computing device if it is determined that the movement has occurred.
4 . The one or more computer-readable storage media of claim 3 , wherein initiating a control action comprises manipulating an object.
5 . The one or more computer-readable storage media of claim 4 further comprising identifying the object for manipulation in accordance with the control action.
6 . The one or more computer-readable storage media of claim 2 further comprising recording the sensor data received from the pointing device.
7 . The one or more computer-readable storage media of claim 6 further comprising receiving an indication of an activation of a button on the pointing device before recording the sensor data.
8 . The one or more computer-readable storage media of claim 6 further comprising:
receiving an indication of an activation of a button on the pointing device; and
receiving an indication of a deactivation of a button on the pointing device,
wherein recording the sensor data occurs between the activation of the button and the deactivation of the button.
9 . The one or more computer-readable storage media of claim 2 , wherein the movement includes an upward, a downward, a rightward, and/or a leftward motion with respect to the object.
10 . The one or more computer-readable storage media of claim 2 , wherein the sensor data includes acceleration data.
11 . The one or more computer-readable storage media of claim 2 further comprising obtaining a plurality of thresholds, wherein each of the thresholds comprises at least one value.
12 . The one or more computer-readable media of claim 11 , wherein identifying whether the threshold is exceeded by the sensor data further comprises determining whether the at least one value in any of the plurality of thresholds is exceeded by the sensor data.
13 . A method for recognizing a movement associated with a pointing device, the method comprising:
referencing at least one threshold value utilized to indicate a motion of the pointing device;
receiving accelerometer data at the computing device, the accelerometer data being output from the pointing device, the accelerometer data including at least one acceleration data value representing an actual motion of the pointing device;
comparing the at least one threshold value to the accelerometer data received from the pointing device;
identifying whether the at least one threshold value is exceeded by the at least one acceleration data value in the accelerometer data; and
manipulating an object on a display screen based on the identification of whether the at least one data value exceeds the at least one threshold value.
14 . The method of claim 13 , wherein the at least one threshold value corresponds to a specific motion of the pointing device.
15 . The method of claim 14 , wherein the specific motion comprises a downward motion, an upward motion, a leftward motion, and/or a rightward motion about or along an axis.
16 . The method of claim 15 , wherein the axis comprises an x-axis, a y-axis, a z-axis, or a combination thereof.
17 . The method of claim 13 , further comprising recording the accelerometer data received from the pointing device.
18 . The method of claim 17 , further comprising receiving an indication of an activation of a button on the pointing device before recording the accelerometer data at the computing device.
19 . The method of claim 18 , further comprising:
receiving an indication at the computing device of an activation of a button on the pointing device; and
receiving an indication at the computing device of a deactivation of a button on the pointing device,
wherein recording the accelerometer data occurs between the activation of the button and the deactivation of the button.
20 . The method of claim 15 , wherein the manipulation corresponds to an upward manipulation, a downward manipulation, a rightward manipulation, and/or a leftward manipulation.