Sequential classification recognition of gesture primitives and window-based parameter smoothing for high dimensional touchpad (HDTP) user interfaces
View Patent ↗A method for classification recognition of gestures and gesture primitives in a touch-based user interface. In an implementation the method comprises receiving tactile image data responsive to data generated from user touch of a user touch interface comprising a sensor array. The tactile image data is processed to create a plurality of numerical values responsive to data generated from the user touch interface. These numerical values are applied to a principle component analysis operation to produce a reduced-dimensionality data vector which is applied to a classifier having a plurality of classifier outputs interpretable as probabilities. The classifier outputs provide likelihoods that an execution gesture is from a collection of pre-defined gestures, and a decision test is used to produce a decision output indicating a gesture outcome useful in user interface applications. The arrangement can recognize single finger “6D” actions of roll, pitch, yaw, left-right, forward-back, and variations in applied pressure.
1. A method for recognition of user gesture primitives in a touch-based user interface using at least one computational processor, the method comprising:
receiving tactile image data responsive to user touch of a user touch interface comprising a tactile sensor array, the user touch comprising a user touch gesture primitives executed over an interval of time;
processing the tactile image data with at least one analysis processing operation to produce a processed data vector comprising a plurality of numerical values responsive to data generated from the tactile image data, the at least one analysis processing operation comprising calculations including deriving first and second moments from the tactile image data and producing the processed data vector based on a centroid calculation on the derived first and second moments; and
further processing the numerical values with a principle component analysis operation to produce a reduced-dimensionality data vector;
applying the reduced-dimensionality data vector to a classifier, the classifier configured to selectively provide a plurality of classifier outputs, the plurality of classifier outputs selectively provided responsive to the reduced-dimensionality data vector, each classifier output representing an associated pre-defined gesture from a set of pre-defined gestures, the classifier further configured to provide an associated probability value for each classifier output;
applying the classifier outputs and the associated probability value for each provided classifier output to a decision test, the decision test producing a decision output,
wherein the plurality of classifier outputs provided by the classifier and the associated probability value for each provided classifier output are responsive to the tactile image data;
wherein the decision output is responsive to the user touch gesture primitives, and
wherein the decision output is used to specify an identified gesture to an external system, the external system comprising software.
2. The method of claim 1 wherein the classifier comprises an artificial neural network.
3. The method of claim 1 wherein the classifier comprises a heuristic.
4. The method of claim 1 wherein the classifier comprises a genetic algorithm.
5. The method of claim 1 wherein the decision output is used as a user interface signal.
6. The method of claim 1 wherein the collection of predefined gestures includes the roll angle change of a single finger on the user touch interface.
7. The method of claim 1 wherein the collection of predefined gestures includes the pitch angle change of a single finger on the user touch interface.
8. The method of claim 1 wherein the collection of pre-defined gestures includes the yaw angle change of a single finger on the user touch interface.
9. The method of claim 1 wherein the collection of pre-defined gestures includes the simultaneous change in the sway (“x”) of the position of a single finger on the user touch interface.
10. The method of claim 1 wherein the numerical values with a principle component analysis operation comprise moments calculated from the tactile image data.
11. The method of claim 1 wherein the numerical values with a principle component analysis operation comprise a roll angle measurement derived from the tactile image data.
12. The method of claim 1 wherein the numerical values with a principle component analysis operation comprise a pitch angle measurement derived from the tactile image data.
13. The method of claim 1 wherein the numerical values with a principle component analysis operation comprise a yaw angle measurement derived from the tactile image data.
14. The method of claim 2 wherein the output layer of the artificial neural network comprises a logistics function element.
15. The method of claim 1 wherein the at least one analysis processing operation produces a plurality of processed data vectors during the time used execution of the intended touch gesture.
16. The method of claim 1 wherein the collection of pre-defined gestures includes the simultaneous change in the surge (“y)” of the position of a single finger on the user touch interface.