Determination of controller three-dimensional location using image analysis and ultrasonic communication

1. A method for determining location of a controller, the method comprising:

capturing image data of a capture area in front of a display;

capturing sound data emitted by the controller in the capture area in front of the display;

calculating a two-dimensional location of the controller based on the captured image data; and

calculating a third dimensional location of the controller based on the captured sound data, wherein calculating the third dimensional location further includes:

measuring a time of flight of the captured sound data; and

calculating a depth within the capture area based on the time of flight of the captured sound data, wherein the depth is an orthogonal measure to both dimensions of the two-dimensional location, wherein operations of the method are executed by a processor.

2. The method as recited in claim 1 , wherein calculating the two-dimensional location of the controller further includes:

detecting a spherical shape within the image data, the spherical shape being associated with a spherical-type section in the controller.

3. The method as recited in claim 2 , wherein the spherical-type section is operable to emit one of a plurality of light colors to assist in identifying the spherical shape in the captured image data.

4. The method as recited in claim 3 , further including:

sending from a processor to the controller a choice of light color from the plurality of light colors, wherein the processor calculates the two-dimensional location, wherein the choice of light color is utilized to identify the spherical shape in the captured image data.

5. The method as recited in claim 1 , wherein calculating the third dimensional location further includes:

measuring and a phase coherence of the captured sound data; and

calculating the depth within the capture area based on the time of flight and the phase coherence of the captured sound data.

6. The method as recited in claim 1 , further including,

presenting in a graphical user interface in the display an object based on the two-dimensional location and the third dimensional location.

7. The method as recited in claim 1 , wherein the sound data is captured with a sound capture device, the method further including:

establishing ultrasound communication between the sound capture device and the controller.

8. The method as recited in claim 1 , further including:

detecting a position of a user head within the captured image data or a position of a user torso within the captured image data.

9. The method as recited in claim 1 , wherein operations of the method are performed by a computer program when executed by one or more processors, the computer program being embedded in a non-transitory computer-readable storage medium.

10. A system for determining location of a controller, comprising:

an image capture device operable to capture image data of a capture area in front of a display;

a sound capture device operable to capture sound data emitted by the controller in the capture area in front of the display; and

a processor in communication with the image capture device and the sound capture device, wherein the processor is operable to measure a time of flight of the captured sound data, wherein the processor is operable to calculate a two-dimensional location of the controller based on the captured image data, and the processor is operable to calculate a third dimensional location of the controller based on the captured sound data and the time of flight, wherein the third dimensional location is orthogonal to both dimensions of the two-dimensional location.

11. The system as recited in claim 10 , wherein the controller includes:

an elongated body with a first end and a second end;

a grip area at about the first end;

an input pad defined between the first end and the second end;

a spherical-type section defined at the second end; and

a sound emitting device.

12. The system as recited in claim 11 , wherein the controller further includes an acoustical chamber, the acoustical chamber defined to direct and receive sound in one or more directions.

13. The system as recited in claim 12 , wherein the controller further includes an inertial analyzer for tracking inertial activity of the controller, wherein the controller is operable to transmit the inertial activity to the system, wherein the processor is further operable to include the inertial activity when calculating the two-dimensional location and the third dimensional location of the controller.

14. The system as recited in claim 10 , wherein the image capture device includes a visible-spectrum camera, wherein the image capture device further includes an infrared camera, wherein the image data from the infrared camera and the visible-spectrum camera are combined in the calculation of the two-dimensional location.

15. The system as recited in claim 10 further including: a speaker connected to the processor, wherein the controller further includes a controller sound capturing device, wherein the controller and the system are operable to establish sound communication via the speaker, the sound capture device, and the controller sound capturing device.

16. The system as recited in claim 10 , wherein the image data corresponds to one of either an red-green-blue (RGB) pixel image data, a black and white pixel image data, or infrared data.

17. The system as recited in claim 10 , wherein the processor is operable to measure a phase coherence of the captured sound data, and the processor is operable to calculate the third dimensional location of the controller within the capture area based on the time of flight and the phase coherence of the captured sound data.

18. A system comprising:

a controller including an image capture device;

one or more light emitters facing a capture area in front of a display, the light emitters located near the display;

a sound capture device operable to capture sound data emitted by the controller when the controller is in the capture area in front of the display, and

a processor in communication with the controller, the one or more light emitters and the sound capture device, wherein the processor is operable to receive from the controller image data captured with the image capture device, wherein the processor determines a position of the light emitters within the image captured data and calculates calculate a two-dimensional location of the controller based on the determined position, wherein the processor calculates a third dimensional location of the controller based on the captured sound data, wherein the processor is operable to measure a time of flight of the captured sound data, and the processor is operable to calculate a depth within the capture area based on the time of flight of the captured sound data.

19. The system as recited in claim 18 , wherein the processor is operable to measure a phase coherence of the captured sound data, and the processor is operable to calculate the depth within the capture area based on the time of flight and the phase coherence of the captured sound data.