Method and system for vision-based interaction in a virtual environment
Method, computer program and system for tracking movement of a subject. The method includes receiving data from a plurality of fixed position sensors comprising a distributed network of time of flight camera sensors to generate a volumetric three-dimensional representation of the subject, identifying a plurality of clusters within the volumetric three-dimensional representation that correspond to features indicative of motion of the subject relative to the fixed position sensors and one or more other portions of the subject, and presenting one or more objects on one or more three dimensional display screens. The plurality of fixed position sensors are used to track motion of the features of the subject to manipulate the volumetric three-dimensional representation to determine interaction of one or more of the features of the subject and one or more of the one or more objects on one or more of the one or more three dimensional display screens.
1. A computer-implemented method for tracking movement of a subject, comprising:
receiving data from a plurality of fixed position sensors comprising a distributed network of time of flight camera sensors to generate a volumetric three-dimensional representation of the subject;
identifying a plurality of clusters within the volumetric three-dimensional representation that correspond to features indicative of motion of the subject relative to the fixed position sensors and one or more other portions of the subject;
presenting one or more objects on one or more three dimensional display screens; and
using the plurality of fixed position sensors to track motion of the features of the subject to manipulate the volumetric three-dimensional representation to determine interaction of one or more of the features of the subject and one or more of the one or more objects on one or more of the one or more three dimensional display screens.
2. The computer-implemented method of claim 1 , wherein each of the one or more three dimensional display screens displays three dimensional information without the use of special eyeware by the subject.
3. The computer-implemented method of claim 1 , wherein each of the one or more three dimensional screens displays three dimensional information employing special eyeware by the subject.
4. The computer-implemented method of claim 1 , further comprising identifying one or more events corresponding to the tracked motion of the subject.
5. The computer-implemented method of claim 4 , wherein identifying one or more events further comprises matching the motion of the subject with an event stored in an event library.
6. The computer-implemented method of claim 5 , wherein the one or more identified events comprises one or more controlling events for controlling a computer.
7. The computer-implemented method of claim 1 , wherein the subject interacts with the one or more objects as they would appear in a real environment.
8. The computer-implemented method of claim 1 , wherein a plurality of the one or more three dimensional display screens is arranged to provide a plurality of different views of the subject.
9. The computer-implemented method of claim 8 , wherein the plurality of the one or more three dimensional display screens is arranged to provide a substantially 360 degree display to the subject.
10. A computer-implemented method for tracking movement of a subject, comprising:
receiving data from a plurality of fixed position sensors comprising a distributed network of time of flight camera sensors to identify a plurality of clusters corresponding to features indicative of motion of the subject relative to the fixed position sensors and one or more other clusters;
presenting one or more objects on one or more three dimensional display screens; and
using the plurality of fixed position sensors to track motion of the features of the subject to determine interaction of one or more of the features of the subject and one or more of the one or more objects on one or more of the one or more three dimensional display screens.
11. The computer-implemented method of claim 10 , further comprising identifying one or more events corresponding to the tracked motion of the subject.
12. The computer-implemented method of claim 11 , wherein identifying one or more events further comprises matching the motion of the subject with an event stored in an event library.
13. The computer implemented method of claim 10 , wherein the tracked motion is used to operate a video game.
14. The computer implemented method of claim 10 , wherein the tracked motion is used in a virtual learning environment.
15. The computer implemented method of claim 10 , wherein the tracked motion is used in a virtual collaboration workspace.
16. A non-transitory computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions when executed by a computer, cause the computer to perform:
receiving data from a plurality of fixed position sensors comprising a distributed network of time of flight camera sensors to generate a volumetric three-dimensional representation of the subject;
identifying a plurality of clusters within the volumetric three-dimensional representation that correspond to features indicative of motion of the subject relative to the fixed position sensors and one or more other portions of the subject;
presenting one or more objects on one or more three dimensional display screens; and
using the plurality of fixed position sensors to track motion of the features of the subject to manipulate the volumetric three-dimensional representation to determine interaction of one or more of the features of the subject and one or more of the one or more objects on one or more of the one or more three dimensional display screens.
17. A system for tracking motion of a subject, comprising:
a plurality of fixed position sensors comprising a distributed network of time of flight camera sensors, at least data received from the plurality of fixed position sensors being used to generate a volumetric three-dimensional representation of the subject;
a processor for identifying a plurality of clusters within the volumetric three-dimensional representation that correspond to features indicative of motion of the subject relative to the fixed position sensors and one or more other portions of the subject; and
one or more three dimensional display screens for presenting one or more objects thereon; and
wherein the plurality of fixed position sensors is used to track motion of the features of the subject to manipulate the volumetric three-dimensional representation, and the processor is used to determine interaction of one or more of the features of the subject and one or more of the one or more objects on the one or more three dimensional display screens.
18. The system of claim 17 , wherein each of the one or more three dimensional display screens displays three dimensional information without the use of special eyeware by the subject.
19. The system of claim 17 , wherein each of the one or more three dimensional screens displays three dimensional information employing special eyeware by the subject.
20. The system of claim 17 , wherein the tracked motion is identified as corresponding to one or more events.
21. The system of claim 17 , wherein one or more of the plurality of fixed position sensors is mounted in a bezel of one or more of the one or more three dimensional displays.
22. The system of claim 17 , wherein one or more of the plurality of fixed position sensors is mounted adjacent one or more of the one or more three dimensional displays.
23. The system of claim 22 , where one or more of the plurality of fixed position sensors is mounted in a wall.
24. The system of claim 17 , wherein one or more of the plurality of fixed position sensors is positioned within a discrete unit separated from each of the one or more three dimensional displays.
25. The system of claim 24 , wherein the discrete unit is mounted adjacent one or more of the one or more three dimensional display screens.
26. The system of claim 24 , wherein the plurality of fixed position sensors comprises a distributed grid of sensors.