System and method for providing an ergonomic three-dimensional, gesture based, multimodal interface for use in flight deck applications

1. A method for operating an aircraft, the method comprising:

detecting, with respect to a sensor volume generated by a sensor, (i) a position and location of a chair from a flight deck dashboard, and (ii) a position and location of an arm-rest from the flight deck dashboard;

retrieving a chair model from a memory device, wherein the chair model comprises a chair coupled to an arm-rest, a height of the arm-rest, and a length of the arm-rest extending past an elbow contact point;

retrieving normalized arm anthropology models comprising (i) a normalized angle between a hand at a neutral position and the arm-rest, (ii) a normalized arm and arm-rest intersection point, and (iii) a normalized arm length from the arm and arm-rest intersection point, from the memory device;

detecting, within the sensor volume, gestures performed by a user's palm and fingers;

generating, based on the chair model, the normalized arm anthropology models, and the position and location of (i) the chair and (ii) the arm-rest, a palm ergonomic movement volume;

generating, based on the palm ergonomic movement volume, a valid interaction volume substantially surrounding the user's palm and fingers, the valid interaction volume being smaller than, and located within, the sensor volume;

recognizing when a gesture performed is within the valid interaction volume and indicates a valid gesture input; and

generating an associated system command based on the valid gesture input.

2. The method of claim 1 , wherein generating a valid interaction volume further comprises:

sensing the position of the user's palm and fingers within the valid interaction volume; and

analyzing at least one of a length of the arm-rest, a height of the arm-rest, an angle between the arm-rest and an arm, a palm end point, a palm bottom joint, an elbow-arm-rest contact point, and a chair location.

3. The method of claim 1 , wherein recognizing when a gesture performed within the valid interaction volume indicates a valid gesture input comprises:

generating a three dimensional gesture interaction model comprising a plurality of rules, wherein the rules comprise one or more valid gesture components;

distinguishing, during a gesture made within the valid interaction volume, static and dynamic features of a palm and fingers;

parsing the static and dynamic features of the palm and fingers; and

comparing the static and dynamic features of the palm and fingers to rules defined in the three dimensional gesture interaction model.

4. The method of claim 3 , wherein the step of comparing comprises:

grouping one or more parsed features into component groups based upon the three dimensional gesture interaction model.

5. The method of claim 4 , further comprising:

obtaining user feedback descriptors from a source of user feedback data;

comparing component groups to user feedback descriptors; and

generating visual and aural user feedback.

6. The method of claim 5 , wherein the step of generating visual feedback comprises:

generating a gesture pointer; and

displaying the gesture pointer on a display unit.

7. The method of claim 5 , wherein the step of generating visual feedback comprises:

generating a three dimensional image representative of the user's palm and fingers based upon the task; and

displaying the three dimensional image on a display unit.

8. The method of claim 5 , wherein the step of generating visual feedback comprises:

generating an image representative of a user interface element with at least one associated affordance; and

displaying the image on a display unit.

9. The method of claim 4 , further comprising:

obtaining system command descriptors from a source of system command data; and

comparing component groups to system command descriptors.

10. A volumetric computing system for an aircraft, comprising:

a chair coupled to an arm-rest;

a sensor generating a predetermined sensor volume;

a display device; and

a processor coupled to the chair, the sensor, and the display device, the processor configured to

(a) detect, within the predetermined sensor volume, (i) a position and location of the chair from a flight deck dashboard, and (ii) a position and location of the arm-rest from the flight deck dashboard;

(b) retrieve a chair model from a memory device, wherein the chair model comprises a chair coupled to an arm rest, a height of the arm-rest, and a length of the arm-rest extending past an elbow contact point;

(c) detect, within the predetermined sensor volume, gestures performed by a user's palm and fingers;

(d) generate, based on the chair model, the position and location of the chair and the arm-rest, and normalized arm anthropology models comprising (i) a normalized angle between a hand at a neutral position and an arm-rest, and (ii) a normalized arm length from an intersection point of an arm and the arm-rest, a valid interaction volume substantially surrounding the user's palm and fingers, the valid interaction volume being smaller than, and located within, the sensor volume;

(e) recognize when a detected gesture performed is within the valid interaction volume and indicates a valid gesture input; and

(f) generate an associated system command based on the valid gesture input.

11. The system of claim 10 , wherein the processor is further configured to:

sense the position of the user's palm and fingers within the predetermined sensor volume; and

analyze at least one of a length of the arm-rest, a height of the arm-rest, an angle between the arm-rest and an arm, a palm end point, a palm bottom joint, an elbow-arm-rest contact point, and a chair location.

12. The system of claim 10 , further comprising:

a source of three dimensional gesture interaction model data, wherein the source of model data is coupled to the processor, and wherein the model data comprises a plurality of rules associated with one or more valid gesture components; and

wherein the processor is further configured to (a) distinguish, during a gesture made within the valid interaction volume, static and dynamic features of a palm and fingers; (b) parse the static and dynamic features of the palm and fingers; and (c) compare the parsed features to rules defined in the three dimensional gesture interaction model.

13. The system of claim 12 , further comprising:

an audible device coupled to the processor; and

a source of user feedback descriptor data coupled to the processor;

wherein the processor is further configured to generate visual feedback on the display device and to generate aural feedback on the audible device; and wherein the generation of the visual and aural feedback are based on the user feedback descriptor data.

14. The system of claim 12 , further comprising:

a source of system command descriptor data coupled to the processor

wherein the processor is further configured to generate system commands based on the user system command descriptor data.

15. A method for operating an aircraft, the method comprising:

positioning, with respect to at least one sensor, an arm-rest that is coupled to a chair;

constructing a predetermined sensor volume based on the position and location of the chair from a flight deck dashboard, the position and location of the arm-rest from the flight deck dashboard, and the at least one sensor;

generating, based on

(a) normalized arm anthropology models comprising (i) a normalized angle between a hand at a neutral position and an arm-rest, and (ii) a normalized arm length from an intersection point of an arm and the arm-rest,

(b) chair models comprising a height of the arm-rest and a length of the arm-rest extending past an elbow contact point, and

(c) the position and location of the chair and arm rest measured from a flight deck dashboard,

relative to the predetermined sensor volume, a valid interaction volume substantially surrounding a user's hand, the valid interaction volume being smaller than, and located within, the predetermined sensor volume;

analyzing static and dynamic movements of a user's palm and fingers within the valid interaction volume;

recognizing when the static and dynamic movements of the user's palm and fingers within the valid interaction volume indicate a valid gesture input; and

generating an associated system command based on the valid gesture input.

16. The method of claim 15 , further comprising:

sorting valid gesture input into at least one of (a) gesture entry point, (b) gesture start, (c) gesture body, and (e) gesture end.

17. The method of claim 16 , wherein the step of sorting further comprises:

evaluating gesture recognition success based upon the gesture body.