Methods and systems for touchless control with a mobile device

1. A device for performing touchless control, comprising:

at least one memory device that stores computer-executable instructions; and

at least one processor configured to access the memory device, wherein the processor is configured to execute the computer-executable instructions to:

receive at least one image from at least one camera;

apply a machine learning technique to the image to determine a first map, the first map comprising a function that generates a correspondence between data associated with the image to a feature space;

determine a second map comprising a representation of a user in the image based on the first map;

determine a third map comprising a vector field that encodes key-point to key-point relationships in the image;

determine a pose of the user based on the second map and the third map by predicting, via the machine learning technique, a body part confidence associated with a portion of a body of the user;

identify a gesture of the user in the image based on the pose of the user;

track the pose of the user and the portion of the body of the user for performing touchless control of the device to generate a historical data of the pose and the portion of the body of the user;

determine, based on the tracked pose, a mapping between the portion of the user's body and an input feature associated with the device;

modify the input feature to a first location based on the mapping to enable an interaction between the user and the device; and

modify the input feature to a second location different from the first location based on the historical data in addition to the mapping.

2. The device of claim 1 , wherein the computer-executable instructions to identify the gesture comprise computer-executable instructions to determine that the gesture has an associated duration exceeding a predetermined threshold.

3. The device of claim 1 , wherein the computer-executable instructions to track the portion of the user's body comprise computer-executable instructions to apply a smoothing filter to reduce noisy measurements during the tracking.

4. The device of claim 1 , further comprising computer-executable instructions to determine a second mapping between a second portion of the user's body and the input feature by applying a mathematical transformation to a representation of the portion of the user's body.

5. The device of claim 4 , further comprising computer-executable instructions to modify the input feature at a different speed or at a different precision based on the second mapping.

6. The device of claim 1 , further comprising computer-executable instructions to:

track the pose of the user at a frame rate;

determine that a value of a refresh rate associated with the device is different than a value of the frame rate; and

modify the input feature using a smoothing function, the smoothing function comprising parameters that are determined based on at least the frame rate and/or the refresh rate.

7. A method for performing touchless control of a device, comprising:

receiving at least one image from at least one camera;

applying a machine learning technique to the image to determine a first map, the first map comprising a function that generates a correspondence between data associated with the image to a feature space;

determining a second map comprising a representation of a user in the image based on the first map;

determining a third map comprising a vector field that encodes key-point to key-point relationships in the image;

determining a pose of the user based on the second map and the third map by predicting, via the machine learning technique, a body part confidence associated with a portion of a body of the user;

identifying a gesture of the user in the image based on the pose of the user;

tracking the pose of the user and the portion of the body of the user for performing touchless control of the device to generate a historical data of the pose and the portion of the body of the user;

determining, based on the tracked pose, a mapping between the portion of the user's body and an input feature associated with the device;

modifying the input feature to a first location based on the mapping to enable an interaction between the user and the device; and

modifying the input feature to a second location different from the first location based on the historical data in addition to the mapping.

8. The method of claim 7 , wherein the identifying the gesture further comprises determining that the gesture has an associated duration exceeding a predetermined threshold.

9. The method of claim 7 , wherein the tracking the portion of the user's body further comprises applying a smoothing filter to reduce noisy measurements during the tracking.

10. The method of claim 7 , further comprising determining a second mapping between a second portion of the user's body and the input feature by applying a mathematical transformation to a representation of the portion of the user's body.

11. The method of claim 10 , further comprising modifying the input feature at a different speed or at a different precision based on the second mapping.

12. The method of claim 7 , further comprising:

tracking the pose of the user at a frame rate;

determining that a value of a refresh rate associated with the device is different than a value of the frame rate; and

modifying the input feature using a smoothing function, the smoothing function comprising parameters that are determined based on at least the frame rate and/or the refresh rate.

13. A non-transitory computer-readable medium storing computer-executable instructions which, when executed by a processor, cause the processor to perform operations for performing touchless control of a device, the operations to:

receive at least one image from at least one camera;

apply a machine learning technique to the image to determine a first map, the first map comprising a function that generates a correspondence between data associated with the image to a feature space;

determine a second map comprising a representation of a user in the image based on the first map;

determine a third map comprising a vector field that encodes key-point to key-point relationships in the image;

determine a pose of the user based on the second map and the third map by predicting, via the machine learning technique, a body part confidence associated with a portion of a body of the user;

identify a gesture of the user in the image based on the pose of the user;

track the pose of the user and the portion of the body of the user for performing touchless control of the device to generate a historical data of the pose and the portion of the body of the user;

determine, based on the tracked pose, a mapping between the portion of the user's body and an input feature associated with the device;

modify the input feature to a first location based on the mapping to enable an interaction between the user and the device; and

modify the input feature to a second location different from the first location based on the historical data in addition to the mapping.

14. The non-transitory computer-readable medium of claim 13 , wherein the computer-executable instructions to track the portion of the user's body comprise computer-executable instructions to apply a smoothing filter to reduce noisy measurements during the tracking.

15. The non-transitory computer-readable medium of claim 13 , further comprising computer-executable instructions to determine a second mapping between a second portion of the user's body and the input feature by applying a mathematical transformation to a representation of the portion of the user's body.

16. The non-transitory computer-readable medium of claim 15 , further comprising computer-executable instructions to modify the input feature at a different speed or at a different precision based on the second mapping.

17. The non-transitory computer-readable medium of claim 13 , further comprising computer-executable instructions to:

track the pose of the user at a frame rate;

determine that a value of a refresh rate associated with the device is different than a value of the frame rate; and

modify the input feature using a smoothing function, the smoothing function comprising parameters that are determined based on at least the frame rate and/or the refresh rate.