Systems and methods of gestural interaction in a pervasive computing environment
The technology disclosed relates to selecting among devices room to interact with. It also relates operating a smart phone with reduced power consumption. It further relates to gesturally interacting with devices that lack gestural responsiveness. The technology disclosed also relates to distinguishing control gestures from proximate non-control gestures in a pervasive three dimensional (3D) sensory space. The technology disclosed further relates to selecting among virtual interaction modalities to interact with.
1. A method of controlling a vocally selected device in a three dimensional (3D) sensory space, the method including:
detecting, using a smart phone, one or more heterogeneous devices that accept motion control commands;
presenting a graphical display of the detected devices;
detecting, using the smart phone, a voice command;
interpreting, using the smart phone, the voice command as selecting one of the detected devices as a vocally selected device;
establishing a communication channel between the smart phone and the vocally selected device;
capturing a series of temporally sequential images of the hand as it moves, using the smart phone, wherein the smart phone is equipped with a motion sensory control device that detects gestures made by a hand moving freely in at least three dimensions in a three dimensional (3D) sensory space;
detecting one or more gestures in the 3D sensory space using the captured series of temporally sequential images;
preprocessing the detected one or more gestures by reducing an amount of data per frame representing the gesture, allowing a reduction in the required bandwidth;
performing motion-capture image analysis on the preprocessed gestures using a mocap program comprising a plurality of modules;
determining a gesture of the hand corresponding to the temporally sequential images; and
controlling, via the communication channel, the vocally selected device based, at least in part, by sending the determined gesture to the vocally selected device.
2. The method of claim 1 , further including:
creating data configured for display by the smart phone that identifies the vocally selected device; and
transmitting the data to the smart phone.
3. The method of claim 1 , further including:
determining that the gesture is a paradigm-setting gesture; and
setting a control paradigm used to preprocess one or more gestures in the 3D sensory space by modifying responsiveness or functionality for controlling the vocally selected device; and
sending the one or more preprocessed gestures to the selected device via the communications channel.
4. The method of claim 3 , wherein
controlling the vocally selected device comprises
controlling responsiveness of the vocally selected device to the one or more preprocessed gestures based at least on the set control paradigm.
5. The method of claim 1 , further including:
modifying the image analysis based, at least in part, on computational capacity of the smart phone.
6. The method of claim 1 , wherein:
the step of detecting, using the smart phone, a voice command is carried out using a microphone in the smart phone.
7. The method of claim 1 , wherein:
the motion sensory control device that detects gestures comprises one or more cameras.