Method and apparatus for tracking of a plurality of subjects in a video game

1. A system for tracking movement of motion features of a plurality of subjects for operating a video game, comprising:

a plurality of fixed position sensors comprising a distributed network of camera sensors including one or more emitted light sources associated with one or more of the one or more camera sensors, the plurality of fixed position sensors capturing data and generating a volumetric three-dimensional representation for a first subject and a second subject;

a processor for identifying one or more motion features of each of the first and second subjects within the three-dimensional representation indicative of motion of one or more motion features of the first subject relative to the fixed position sensors, one or more other portions of the first subject, and one or more portions of the second subject;

one or more three dimensional displays for presenting the generated volumetric three-dimensional representation for the first subject and for the second subject; and

the processor further using the plurality of fixed position sensors to track motion of the one or more motion features of the first subject to determine interaction of the one or more motion features of the first subject and one or more portions of the second subject on one or more of the one or more three dimensional displays.

2. The system of claim 1 , wherein the one or more three dimensional displays presents one or more objects; and

wherein the plurality of fixed position sensors track motion of the one or more motion features of the first subject to determine interaction of the one or more motion features of the first subject and the one or more objects on one or more of the one or more three dimensional displays.

3. The system of claim 1 , the processor further identifying one or more recognized events within the video game corresponding to the tracked motion of the first subject.

4. The system of claim 3 , wherein identifying one or more events further comprises matching the motion of the first subject with an event stored in an event library associated with the video game.

5. The system of claim 4 , wherein the processor further generates new events for the library associated with the video game taking into account one or more of the anatomical constraints of the first subject, sensor resolution, pixel depth and a total number of sensors.

6. The system of claim 1 , wherein the processor further:

acquires an image of the face of the first and second subjects playing the video game; and

integrates a non-photo-real avatar resembling the acquired image of the first and second subjects into first and second characters in the video game.

7. The system of claim 6 , wherein the processor further warps each of the non-photo-real avatars to best match a non-photo-real color space associated with the video game.

8. The system of claim 6 , wherein each of the non-photo-real avatars changes according to one or more game conditions.

9. The system of claim 8 , wherein the one or more game conditions comprises a change in skill level of a character.

10. The system of claim 8 , wherein the changing of the non-photo-real avatar represents a nonlinear transformation.

11. The system of claim 10 , wherein the nonlinear transformation comprises eigen faces fusion between the avatar and the subject.

12. The system of claim 10 , wherein the nonlinear transformation comprises landmarking between the avatar and the subject.

13. The system of claim 1 , wherein the one or more three dimensional displays comprise autostereoscopic displays.

14. The system of claim 1 , wherein the one or more three dimensional displays employ a three dimensional viewing device.

15. The system of claim 14 , wherein the three dimensional viewing device comprises three dimensional viewing glasses.

16. The system of claim 1 , wherein the fixed position sensors are time-of-flight sensors.

17. A system for tracking motion of a subject during operation of a video game, comprising:

a distributed network of camera sensors employing one or more emitted light sources associated with one or more of the one or more camera sensors, at least data received from the camera sensors being used to generate a volumetric three-dimensional representation of each of a first and second subject;

a processor for identifying one or more motion features of each of the first and second subjects within the three-dimensional representation indicative of motion of one or more motion features of the first subject relative to the fixed position sensors, one or more other portions of the first subject and one or more portions of the second subject;

one or more three dimensional displays for representing the generated volumetric three-dimensional representations for each of the first and second subjects; and

wherein the plurality of fixed position sensors are used to track motion of the one or more motion features of the first subject to determine interaction of the one or more motion features of the first subject and one or more portions of the second subject on one or more of the one or more three dimensional displays.

18. The system of claim 17 , wherein the plurality of fixed position sensors comprises a distributed grid of sensors.

19. The system of claim 18 , wherein the distributed grid of sensors further comprises a plurality of light sources.

20. The system of claim 17 , wherein a representation of the volumetric three-dimensional representation for each of the first and second subjects is displayed on one or more of the three dimensional displays, wherein each of the volumetric three-dimensional representations move according to instructions generated by a software development kit that links the motion features to a skeletal structure.

21. The system of claim 20 , wherein the one or more displays display interaction between the representations of the volumetric three-dimensional representations and one or more objects within the video game.

22. The system of claim 17 , wherein the processor further identifies one or more recognized events within the video game corresponding to the tracked motion of the first subject.

23. The system of claim 22 , wherein identifying one or more events by the processor further comprises matching the motion of the first subject with an event stored in an event library associated with the video game.

24. The system of claim 23 , wherein the processor generates new events for the library associated with the video game taking into account one or more of the anatomical constraints of the subject, sensor resolution, pixel depth, location of light source, and a total number of sensors.

25. The system of claim 17 , wherein the processor acquires an image of the face of the first and second subjects playing the video game in accordance with at least data received from the camera sensor, and integrates a non-photo-real avatar resembling the acquired image of the first and second subjects into a character in the video game.

26. The system of claim 25 , wherein the non-photo-real avatars are warped by the processor to best match a non-photo-real color space associated with the video game.

27. The system of claim 25 , wherein the processor changes the non-photo-real avatars according to one or more game conditions.

28. The system of claim 27 , wherein the one or more game conditions comprises a change in skill level of a character controlled by one of the first and second subjects.

29. The system of claim 27 , wherein the changing of the non-photo-real avatar represents a nonlinear transformation.

30. The system of claim 29 , wherein the nonlinear transformation comprises eigen faces fusion between the avatar and the corresponding one of the first and second subjects.

31. The system of claim 29 , wherein the nonlinear transformation comprises landmarking between the avatar and corresponding ones of the first and the second subjects.

32. The system of claim 17 , wherein the one or more three dimensional displays comprise autostereoscopic displays.

33. The system of claim 17 , wherein the one or more three dimensional displays employ a three dimensional viewing device.

34. The system of claim 33 , wherein the three dimensional viewing device comprises three dimensional viewing glasses.

35. The system of claim 17 , wherein the fixed position sensors are time-of-flight sensors.

36. A non-transitory computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions when executed by a computer coupled to a plurality of fixed position sensors for receiving data and one or more three dimensional displays for presenting one or more objects thereon, causes the computer to track movement of motion features of a subject operating a video game by performing the steps of:

receiving data from the plurality of fixed position sensors comprising a distributed network of camera sensors employing one or more emitted light sources associated with one or more of the one or more camera sensors to generate a volumetric three-dimensional representation for a first subject and a second subject;

identifying one or more motion features of each of the first and second subjects within the three-dimensional representation indicative of motion of one or more motion features of the first subject relative to the fixed position sensors one or more other portions of the first subject, and one or more portions of the second subject;

presenting the generated volumetric three-dimensional representation for the first subject and for the second subjects on the one or more three dimensional displays; and

using the plurality of fixed position sensors to track motion of the one or more motion features of the first subject to determine interaction of the one or more motion features of the first subject and one or more portions of the second subject on one or more of the one or more three dimensional displays.

37. The computer-readable medium of claim 36 , having stored thereon additional instructions, the additional instructions when executed by the computer coupled to the plurality of fixed position sensors for receiving data and one or more three dimensional displays for presenting one or more objects thereon, cause the computer to further perform the steps of:

presenting one or more objects on the one or more three dimensional displays; and

using the plurality of fixed position sensors to track motion of the one or more motion features of the first subject to determine interaction of the one or more motion features of the first subject and the one or more objects on one or more of the one or more three dimensional displays.

38. The computer-readable medium of claim 36 , having stored thereon additional instructions, the additional instructions when executed by a computer coupled to the plurality of fixed position sensors for receiving data and one or more three dimensional displays for presenting one or more objects thereon, cause the computer to further perform the step of identifying one or more recognized events within the video game corresponding to the tracked motion of the first subject.

39. The computer-readable medium of claim 38 , wherein identifying one or more events further comprises matching the motion of the first subject with an event stored in an event library associated with the video game.

40. The computer-readable medium of claim 39 , having stored thereon additional instructions, the additional instructions when executed by a computer coupled to the plurality of fixed position sensors for receiving data and one or more three dimensional displays for presenting one or more objects thereon, cause the computer to further perform the step of generating new events for the library associated with the video game taking into account anatomical constraints of the first subject, sensor resolution, pixel depth and a total number of sensors.

41. The computer-readable medium of claim 36 , having stored thereon additional instructions, the additional instructions when executed by a computer coupled to the plurality of fixed position sensors for receiving data and one or more three dimensional displays for presenting one or more objects thereon, cause the computer to further perform the steps of:

acquiring an image of the face of the first and second subjects playing the video game; and

integrating a non-photo-real avatar resembling the acquired image of the first and second subjects into first and second characters in the video game.

42. The computer-readable medium of claim 41 , wherein each of the non-photo-real avatars is warped to best match a non-photo-real color space associated with the video game.

43. The computer-readable medium of claim 41 , wherein each of the non-photo-real avatars changes according to one or more game conditions.

44. The computer-readable medium of claim 43 , wherein the one or more game conditions comprises a change in skill level of a character.

45. The computer-readable medium of claim 43 , wherein the changing of the non-photo-real avatar represents a nonlinear transformation.

46. The computer-readable medium of claim 45 , wherein the nonlinear transformation comprises eigen faces fusion between the avatar and the subject.

47. The computer-readable medium of claim 45 , wherein the nonlinear transformation comprises landmarking between the avatar and the subject.

48. The computer-readable medium of claim 36 , wherein the one or more three dimensional displays comprise autostereoscopic displays.

49. The computer-readable medium of claim 36 , wherein the one or more three dimensional displays employ a three dimensional viewing device.

50. The computer-readable medium of claim 49 , wherein the three dimensional viewing device comprises three dimensional viewing glasses.

51. The computer-readable medium of claim 36 , wherein the fixed position sensors are time-of-flight sensors.