Gesture-based user interface device and method
A user interface device including memory configured to store instructions and at least one hardware processor configured to execute the instructions to obtain, from at least one sensor, sensor data corresponding to a user gesture, determine a user interface gesture that matches the sensor data, obtain parameters for the determined interface gesture, and upon obtaining the parameters for the determined interface gesture, use the sensor data and the parameters for the determined interface gesture to estimate the user gesture in a user interface. Also provided is a method.
1. A device comprising:
memory configured to store instructions, executable by a processor, and to store parameters; and
at least one hardware processor configured to execute the instructions to perform steps of a method comprising:
obtaining, from at least one sensor, sensor data corresponding to a captured user gesture; and
upon determining that not all the sensor data is reliable:
determining a stored user interface gesture that matches the sensor data;
obtaining parameters for the determined stored user interface gesture learnt from a present user, wherein:
in case no parameters for the determined stored user interface gesture learnt from the present user are obtained, obtaining default parameters for the determined stored user interface gesture;
in case no default parameters for the determined stored user interface gesture are obtained, using the sensor data in the user interface; and
enhancing reliable parts of the sensor data with the parameters for the determined stored user interface gesture by replacing unreliable sensor data with data based on the parameters to estimate the user gesture in a user interface.
2. The device of claim 1 , wherein the determining the stored user interface gesture that matches the sensor data further comprises:
determining the stored user interface gesture with a highest similarity score to the sensor data.
3. The device of claim 1 , wherein the data based on the parameters is further obtained from reliable sensor data.
4. The device of claim 3 , wherein the sensor data comprises node coordinates and relative node coordinates of the user interface gesture and wherein the reliable sensor data comprises reliable coordinates for at least one node and wherein the data based on the parameters is obtained by adding relative node coordinates to the reliable coordinates for the at least one node.
5. The device of claim 1 , wherein the estimated user gesture is used in the user interface for at least one of determining interaction with an object in the user interface and rendering of the user gesture in the user interface.
6. The device of claim 1 , wherein the at least one hardware processor is configured to determine whether all the sensor data is reliable.
7. The device of claim 1 , wherein the at least one hardware processor is further configured to, upon determining that all the sensor data is reliable, estimating the user gesture in the user interface based only on the sensor data.
8. A method implemented in at least one hardware processor of a device, the method comprising:
obtaining, from at least one sensor, sensor data corresponding to a captured user gesture; and
upon determining that not all the sensor data is reliable:
determining a stored user interface gesture that matches the sensor data;
obtaining parameters for the determined stored user interface gesture learnt from a present user, wherein:
in case no parameters for the determined stored user interface gesture learnt from the present user are obtained, obtaining default parameters for the determined stored user interface gesture;
in case no default parameters for the determined stored user interface gesture are obtained, using the sensor data in the user interface; and
enhancing reliable parts of the sensor data with the parameters for the determined stored user interface gesture by replacing unreliable sensor data with data based on the parameters to estimate the user gesture in a user interface.
9. The method of claim 8 , wherein determining the stored user interface gesture that matches the sensor data comprises determining the stored user interface gesture with a highest similarity score to the sensor data.
10. The method of claim 8 , further comprising upon determining that all the sensor data is reliable, estimating the user gesture in the user interface based only on the sensor data.
11. A non-transitory computer-readable storage medium storing instructions that, when executed, cause at least one hardware processor to perform a method of claim 8 .