Gestures, interactions, and common ground in a surface computing environment
Aspects relate to detecting gestures that relate to a desired action, wherein the detected gestures are common across users and/or devices within a surface computing environment. Inferred intentions and goals based on context, history, affordances, and objects are employed to interpret gestures. Where there is uncertainty in intention of the gestures for a single device or across multiple devices, independent or coordinated communication of uncertainty or engagement of users through signaling and/or information gathering can occur.
1. A method performed by one or more processors that are configured with executable instructions to perform acts comprising:
receiving, by a first computing device, a gesture input from a touch-sensitive input device;
in response to determining that the gesture input comprises:
an undefined gesture that is excluded from a set of gestures stored by the first computing device, receiving a set of user gestures from a second computing device;
in response to determining that the gesture input is included in the set of user gestures,
performing an action corresponding to the gesture input; and
in response to determining that a second gesture input received within a predetermined time period comprises a duplicate gesture, ignoring the duplicate gesture, wherein the gesture input and the second gesture input are provided to a same software application being executed by the first computing device.
2. The method of claim 1 , the method further comprising:
determining a context associated with the gesture input; and
determining the action based at least partly on the context associated with the gesture input.
3. The method of claim 1 , the method further comprising:
providing a visual representation of the gesture input.
4. The method of claim 1 , the method further comprising:
in response to determining that the gesture input is included in a set of common gestures, determining a predetermined action corresponding to the gesture input based on the set of common gestures; and
performing the predetermined action.
5. The method of claim 1 , the method further comprising:
identifying the second computing device as being associated with a user that generated the gesture input;
sending a request to the second computing device to provide the set of user gestures; and
receiving the set of user gestures from the second computing device.
6. The method of claim 1 , further comprising:
in response to determining that a third gesture input is excluded from the set of user gestures, prompting a user to select an additional action to perform when the third gesture is received.
7. A device comprising:
a touch-screen device;
one or more processors;
one or more memory devices to store instructions that are executable by the one or more processors to perform operations comprising:
receiving a gesture input from the touch-screen device;
in response to determining that the gesture input is excluded from a set of gestures stored in the one or more memory devices, receiving a set of user gestures from a second computing device;
determining an action corresponding to the gesture input based at least partly on the set of user gestures;
performing the action corresponding to the gesture input; and
in response to determining that a second gesture input received from the touch-screen device within a predetermined time period after receiving the gesture input comprises a duplicate gesture of the gesture input, ignoring the duplicate gesture, wherein the gesture input and the second gesture input are provided to a same software application being executed by the device.
8. The device of claim 7 , the operations further comprising:
displaying a visual representation of the gesture.
9. The device of claim 7 , the operations further comprising:
determining a context associated with the gesture input; and
determining the action corresponding to the gesture input based at least partly on the context associated with the gesture input.
10. The device of claim 7 , the operations further comprising;
determining that a third gesture input is included in the set of gestures; and
automatically performing a predetermined action corresponding to the third gesture input.
11. The device of claim 7 , the operations further comprising:
determining that the second computing device is associated with a user that generated the gesture input; and
receiving the set of user gestures from the second computing device in response to sending, from the device to the second computing device, a request to send the set of user gestures.
12. The device of claim 7 , the operations further comprising:
in response to determining that a third gesture is excluded from the set of user gestures, receiving user input selecting a third action to perform when the third gesture is received.
13. The device of claim 7 , the operations further comprising:
tracking a gaze of a user that is using the device;
identifying, based on the gaze of the user, an item displayed on a display device; and
performing the action to the item.
14. A method comprising:
detecting, by a first computing device, a gesture performed on a touch-screen display of the first computing device;
determining that the gesture is excluded from a set of gestures stored on the first computing device;
requesting a set of user gestures from a second computing device;
performing an action corresponding to the gesture based at least partly on the set of user gestures;
determining that a second gesture input received from the touch-screen display within a predetermined time period comprises a duplicate gesture, wherein the gesture and the second gesture are sent to a same software application executing on the first computing device; and
ignoring the duplicate gesture.
15. The method of claim 14 , further comprising:
determining a context associated with the gesture; and
determining the action corresponding to the gesture based at least partly on the context associated with the gesture.
16. The method of claim 14 , further comprising:
determining that a third gesture is included in the set of gestures; and
performing a pre-determined action corresponding to the third gesture.
17. The method of claim 14 , further comprising:
tracking a gaze of a user that input the gesture;
determining, based on the gaze of the user, a context associated with the gesture; and
performing a pre-determined action based at least partly on the context.
18. The method of claim 14 , further comprising:
determining that the second computing device is associated with a user that input the gesture;
sending a request to the second computing device; and
receiving the set of user gestures from the second computing device.
19. The method of claim 14 , further comprising:
in response to determining that a fourth gesture is excluded from the set of user gestures, receiving user input selecting a fourth action;
associating the fourth action with the fourth gesture; and
storing the fourth action and the fourth gesture in the set of user gestures.
20. The method of claim 14 , further comprising:
creating the set of gestures by training a classifier implemented using a support vector machine.