Architecture for controlling a computer using hand gestures
Architecture for implementing a perceptual user interface. The architecture comprises alternative modalities for controlling computer application programs and manipulating on-screen objects through hand gestures or a combination of hand gestures and verbal commands. The perceptual user interface system includes a tracking component that detects object characteristics of at least one of a plurality of objects within a scene, and tracks the respective object. Detection of object characteristics is based at least in part upon image comparison of a plurality of images relative to a course mapping of the images. A seeding component iteratively seeds the tracking component with object hypotheses based upon the presence of the object characteristics and the image comparison. A filtering component selectively removes the tracked object from the object hypotheses and/or at least one object hypothesis from the set of object hypotheses based upon predetermined removal criteria.
1. A system that facilitates a user interface, comprising:
memory;
a tracking component configured to detect a first object within a scene and tracks the first object, detection of the first object based at least in part upon image comparison of a plurality of images relative to a course mapping of the images;
a seeding component configured to iteratively seed the tracking component with a set of object hypotheses based at least in part upon a presence of the first object and the image comparison; and
a filtering component configured to selectively remove a first object hypothesis from consideration based at least in part on a determination that the first object hypothesis has moved within a threshold distance of an other object hypothesis, the filtering component further configured to remove a second object hypothesis from consideration based at least in part on a motion threshold associated with an average movement over a window of time, wherein the second object hypothesis is removed from consideration when a duration of a pause in motion associated with the second object hypothesis exceeds an average time parameter.
2. The system of claim 1 , wherein the first object is a hand.
3. The system of claim of claim 1 , wherein the tracking component identifies object characteristics, which include object features and object orientation.
4. The system of claim 3 , wherein the object characteristics include hand movement, hand orientation, and hand pose.
5. The system of claim 3 , wherein the object characteristics include finger count, finger orientation.
6. A computer readable storage medium that is not a signal and that stores instructions for performing a method to facilitate a user interface, the method comprising:
detecting a first object within a scene, wherein detection of the first object is based at least in part upon image comparison of a plurality of images relative to a course mapping of the images;
tracking the first object;
seeding a set of object hypotheses based at least in part upon a presence of the first object and the image comparison;
filtering the set of object hypotheses to remove a first object hypothesis from consideration based at least in part on a determination that the first object hypothesis has moved within a threshold distance of an other object hypothesis; and
filtering the set of object hypotheses to remove a second object hypothesis from consideration based at least in part on a motion threshold associated with an average movement over a window of time, wherein the second object hypothesis is removed from consideration when a duration of a pause in motion associated with the second object hypothesis exceeds an average time parameter.
7. The computer readable storage medium of claim 6 , wherein the first object is identified as a dominant hand, wherein the dominant hand provides gestures that control a computer system.
8. The computer readable storage medium of claim 6 , wherein object characteristics of the first object are extracted.
9. The computer readable storage medium of claim 8 , wherein the object characteristics include finger count and finger orientation.
10. The computer readable storage medium of claim 8 , wherein the object characteristics include hand movement, hand orientation, and hand pose.
11. A method to facilitate a user interface, the method comprising:
detecting a first object within a scene, wherein detection of the first object is based at least in part upon image comparison of a plurality of images relative to a course mapping of the images;
tracking the first object;
seeding a set of object hypotheses based at least in part upon a presence of the first object and the image comparison;
filtering the set of object hypotheses to remove a first object hypothesis from consideration based at least in part on a determination that the first object hypothesis has moved within a threshold distance of an other object hypothesis; and
filtering the set of object hypotheses to remove a second object hypothesis from consideration based at least in part on a motion threshold associated with an average movement over a window of time, wherein the second object hypothesis is removed from consideration when a duration of a pause in motion associated with the second object hypothesis exceeds an average time parameter.
12. The method of claim 11 , wherein the first object is identified as a dominant hand, wherein the dominant hand provides gestures that control a computer system.
13. The method of claim 11 , wherein object characteristics of the first object are extracted.
14. The method of claim 13 , wherein the object characteristics include finger count and finger orientation.
15. The method of claim 13 , wherein the object characteristics include hand movement, hand orientation, and hand pose.