Mapping movements of a hand-held controller to the two-dimensional image plane of a display screen

1. A method for use in controlling a system, comprising the steps of:

tracking three-dimensional movements of a hand-held controller for the system by mixing outputs from two or more different types of sensors;

providing a first ratio to change weights that are applied to the outputs of the two or more different types of sensors;

dynamically changing the weights that are applied to the outputs of the two or more different types of sensors in response to changes in game state of a game application;

mapping the tracked three-dimensional movements of the handheld controller onto a two-dimensional image plane of a screen of a display for the system;

displaying one or more images on the two-dimensional image plane of the screen based on the mapping of the tracked three-dimensional movements of the handheld controller; and

providing a second ratio to change a gearing between the three-dimensional movements of the hand-held controller and a location of the displayed one or more images, wherein the second ratio is changed dynamically in response to user input.

2. A method in accordance with claim 1 , wherein the one or more images comprise a pointer for use in selecting items displayed on the screen.

3. A method in accordance with claim 1 , wherein the one or more images comprise a set of crosshairs.

4. A method in accordance with claim 1 , wherein the step of tracking comprises the step of:

receiving on an image plane of a camera a projection of one or more detectable elements established on the hand-held controller.

5. A method in accordance with claim 4 , wherein the step of tracking further comprises the step of:

analyzing movements and deformities in the projection of the one or more detectable elements.

6. A method in accordance with claim 1 , wherein the step of tracking comprises the step of:

sensing an orientation of the hand-held controller using an inertial sensor.

7. A method in accordance with claim 1 , wherein the step of tracking comprises the step of:

sensing an orientation of the hand-held controller using acoustic signals.

8. A method in accordance with claim 1 , wherein the step of tracking comprises the step of:

sensing an orientation of the hand-held controller by sensing an output of a linear infrared array.

9. An apparatus for use in controlling a system, comprising:

two or more different types of sensors configured to track three-dimensional movements of a hand-held controller for the system, wherein the three-dimensional movements of the hand-held controller are tracked by mixing outputs from the two or more different types of sensors, and wherein a first ratio is provided to change weights that are applied to the outputs of the two or more different types of sensors; and

a processor configured to dynamically change the weights that are applied to the outputs of the two or more different types of sensors in response to changes in game state of a game application, map the tracked three-dimensional movements of the handheld controller onto a two-dimensional image plane of a screen of a display for the system, display one or more images on the two-dimensional image plane of the screen based on the mapping of the tracked three-dimensional movements of the handheld controller, and provide a second ratio to change a gearing between the three-dimensional movements of the hand-held controller and a location of the displayed one or more images, wherein the second ratio is changed dynamically in response to user input.

10. An apparatus in accordance with claim 9 , wherein the one or more images comprise a pointer for use in selecting items displayed on the screen.

11. An apparatus in accordance with claim 9 , wherein the one or more images comprise a set of crosshairs.

12. An apparatus in accordance with claim 9 , wherein the two or more different types of sensors comprises a camera that receives on an image plane thereof a projection of one or more detectable elements established on the hand-held controller.

13. An apparatus in accordance with claim 12 , wherein the processor is further configured to track by analyzing movements and deformities in the projection of the one or more detectable elements.

14. An apparatus in accordance with claim 9 , wherein the two or more different types of sensors comprises an inertial sensor that is configured to sense an orientation of the hand-held controller.

15. An apparatus in accordance with claim 9 , wherein the two or more different types of sensors comprises means for sensing an orientation of the hand-held controller using acoustic signals.

16. An apparatus in accordance with claim 9 , wherein the two or more different types of sensors comprises a linear infrared array that is configured to sense an orientation of the hand-held controller.

17. An apparatus in accordance with claim 9 , further comprising a mixer for mixing the outputs from the two or more different types of sensors.

18. An apparatus for use in controlling a system, comprising:

means for tracking three-dimensional movements of a hand-held controller for the system by mixing outputs from two or more different types of sensors;

means for providing a first ratio to change weights that are applied to the outputs of the two or more different types of sensors;

means for dynamically changing the weights that are applied to the outputs of the two or more different types of sensors in response to changes in game state of a game application;

means for mapping the tracked three-dimensional movements of the handheld controller onto a two-dimensional image plane of a screen of a display for the system;

means for displaying one or more images on the two-dimensional image plane of the screen based on the mapping of the tracked three-dimensional movements of the handheld controller; and

means for providing a second ratio to change a gearing between the three-dimensional movements of the hand-held controller and a location of the displayed one or more images, wherein the second ratio is changed dynamically in response to user input.

19. An apparatus in accordance with claim 18 wherein the one or more images comprise a pointer for use in selecting items displayed on the screen.

20. An apparatus in accordance with claim 18 , wherein the one or more images comprise a set of crosshairs.

21. An apparatus in accordance with claim 18 , wherein the means for tracking comprises:

means for receiving on an image plane of a camera a projection of one or more detectable elements established on the hand-held controller.

22. An apparatus in accordance with claim 21 , wherein the means for tracking further comprises:

means for analyzing movements and deformities in the projection of the one or more detectable elements.

23. An apparatus in accordance with claim 18 wherein the means for tracking comprises:

means for sensing an orientation of the hand-held controller using an inertial sensor.

24. An apparatus in accordance with claim 18 , wherein the means for tracking comprises:

means for sensing an orientation of the hand-held controller using acoustic signals.

25. An apparatus in accordance with claim 18 , wherein the means for tracking comprises:

means for sensing an orientation of the hand-held controller by sensing an output of a linear infrared array.

26. An apparatus in accordance with claim 18 , further comprising:

means for mixing the outputs from the two or more different types of sensors.

27. A computer readable storage medium storing one or more computer readable programs configured to cause a processor based system to execute steps comprising:

tracking three-dimensional movements of a hand-held controller for a system by mixing outputs from two or more different types of sensors;

providing a first ratio to change weights that are applied to the outputs of the two or more different types of sensors;

dynamically changing the weights that are applied to the outputs of the two or more different types of sensors in response to changes in game state of a game application;

mapping the tracked three-dimensional movements of the handheld controller onto a two-dimensional image plane of a screen of a display for the system;

displaying one or more images on the two-dimensional image plane of the screen based on the mapping of the tracked three-dimensional movements of the handheld controller;

and providing a second ratio to change a gearing between the three-dimensional movements of the hand-held controller and a location of the displayed one or more images, wherein the second ratio is changed dynamically in response to user input.