Multipurpose controllers and methods
Method and apparatus is disclosed for a user to communicate with an electronic device. A processor receives user intention actions comprising facial expression (FE) information indicative of facial expressions and body information indicative of motion or position of one or more body parts of the user. When the FE or body information crosses a first level, the processor starts generating first signals based on the FE or body information to communicate with the electronic device. When the FE or body information crosses a second level, the processor can end generation of the first signals or modify the first signals. An image processing or eye gaze tracking system can provide some FE information or body information. The signals can modify attributes of an object of interest. Use of thresholds that are independent of sensor position or orientation with respect to the user's body are also disclosed.
1. A system for detecting facial expressions, the system comprising:
a processor configured to receive facial expression information (FE Info) indicative of one or more facial expressions of a user; and
a first sensor configured to provide at least a portion of said FE Info based on motion of a part of user's skin with respect to said first sensor;
wherein said processor is configured to generate command signals based on said FE Info;
wherein said part of user's skin remains substantially at a constant distance from said first sensor during said motion; and
wherein said first sensor is configured to be worn on the user's face or head.
2. The system of claim 1 , wherein said command signals modify an object of interest ( 00 I) affected by the system.
3. The system of claim 2 , wherein said first sensor comprises a radiation source and an image sensor, wherein said image sensor is configured to image said part of user's skin.
4. The system of claim 3 , further comprising:
a second sensor configured to provide at least one of head information (Head Info) indicative of motion or position of the user's head and eye gaze information (Eye Info) indicative of eye gaze of the user;
wherein said processor is further configured to receive said at least one of said Head Info and said Eye Info;
wherein said command signals are further based on said at least one of said Head Info and said Eye Info.
5. The system of claim 3 , wherein said command signals comprise one or more selection signals.
6. The system of claim 3 , further comprising one or more display mechanisms.
7. The system of claim 6 , wherein said OOI is a graphical object displayed using said one or more display mechanisms.
8. The system of claim 7 , wherein said first sensor provides said at least a portion of said FE Info only when said part of user's skin is in contact with said first sensor.
9. The system of claim 1 , wherein said first sensor provides said at least a portion of said FE Info only when said part of user's skin is in contact with said first sensor.
10. A non-transitory computer readable medium comprising one or more programs configured to be executed by one or more processors to detect one or more facial expressions of a user, said one or more programs causing performance of a method comprising:
receiving facial expression information (FE Info) indicative of motion of a part of the user's skin with respect to a first sensor; and
generating command signals based on said FE Info;
wherein said first sensor is configured to provide at least a portion of said FE Info based on said motion;
wherein said part of user's skin remains substantially at a constant distance from said first sensor during said motion; and
wherein said first sensor is configured to be worn on the user's face or head.
11. The non-transitory computer readable medium of claim 10 , wherein said command signals modify an object of interest ( 00 I) affected by the system.
12. The non-transitory computer readable medium of claim 10 , wherein said first sensor comprises a radiation source and an image sensor, wherein said image sensor is configured to image said part of user's skin.
13. The non-transitory computer readable medium of claim 12 , wherein the method further comprises:
receiving at least one of head information (Head Info) indicative of motion or position of the user's head and eye gaze information (Eye Info) indicative of eye gaze of the user;
wherein said at least one of said Head Info and said Eye Info is provided by a second sensor; and
wherein said command signals are further generated based on said at least one of said Head Info and said Eye Info.
14. The non-transitory computer readable medium of claimer 12 , wherein said command signals comprise one or more selection signals.
15. The non-transitory computer readable medium of claim 12 , wherein said OOI is a graphical object displayed using one or more display mechanisms.
16. The non-transitory computer readable medium of claim 15 , wherein said first sensor provides said at least a portion of said FE Info only when said part of user's skin is in contact with said first sensor.
17. The non-transitory computer readable medium of claim 10 , wherein said first sensor provides said at least a portion of said FE Info only when said part of user's skin is in contact with said first sensor.
18. An apparatus for detecting facial expressions, the apparatus comprising:
memory;
a processor configured to receive facial expression information (FE Info) indicative of one or more facial expressions of a user; and
a first sensor configured to provide at least a portion of said FE Info based on motion of a part of user's skin with respect to said first sensor;
wherein said part of user's skin remains substantially at a constant distance from said first sensor during said motion;
wherein said first sensor is configured to be worn on the user's face or head; and
wherein said processor generates command signals based on said FE Info.
19. The apparatus of claim 18 , wherein said command signals modify an object of interest ( 00 I) affected by an electronic device.
20. The apparatus of claim 19 , wherein said first sensor comprises a radiation source and an image sensor, wherein said image sensor is configured to image said part of user's skin.
21. The apparatus of claim 20 , further comprising:
a second sensor configured to provide at least one of head information (Head Info) indicative of motion or position of the user's head and eye gaze information (Eye Info) indicative of eye gaze of the user;
wherein said processor is further configured to receive said at least one of said Head Info and said Eye Info; and
wherein said command signals are further based on said at least one of said Head Info and said Eye Info.
22. The apparatus of claim 20 , wherein said command signals comprise one or more selection signals.
23. The apparatus of claim 20 , further comprising one or more display mechanisms.
24. The apparatus of claim 23 , wherein said OOI is a graphical object displayed using said one or more display mechanisms.
25. The apparatus of claim 24 , wherein said first sensor provides said at least a portion of said FE Info only when said part of user's skin is in contact with said first sensor.
26. The apparatus of claim 20 , wherein said apparatus is configured to be a part of the electronic device.
27. The apparatus of claim 18 , wherein said first sensor provides said at least a portion of said FE Info only when said part of user's skin is in contact with said first sensor.