Touch interaction detection
Various implementations disclosed herein facilitate identify touch interactions based on sensor data from sensors that sense skin deformation of a finger while the finger performs the touch interaction. In some implementations, a processor performs a method to identify a touch interaction by executing instructions stored in a computer-readable medium. The method obtains deformation data from a plurality of sensors. The sensors are positioned adjacent to a plurality of portions of a finger and configured to detect deformations of the plurality of portions of the finger over time. The method detects an event (e.g., make/break events or tap events) based on the deformation data. The event detection may be based on estimates of displacement and/or rate of displacement. The method identifies a touch interaction (e.g., pick & place (long pinches), quick pinches, taps on hard surfaces, swiping on a surface, double tap) based on the detected event.
1 . A method comprising:
at a processor:
obtaining a plurality of channels of deformation data from a plurality of sensors on a device, wherein the sensors are positioned adjacent to a plurality of portions of a finger to detect an amount of change in position of portions of skin corresponding deformations of the plurality of portions of the finger over time;
detecting events based on the deformation data, wherein the events are detected by analyzing time segments of data from the plurality of channels and identifying characteristics of the time segments to distinguish different types of touch interactions, wherein detecting the events comprising:
detecting a first event during which the finger makes contact with a surface by detecting when the finger changes from a first state of not being in contact with the surface to a second state of being in contact with the surface; and
detecting a second event, following the first event, during which the finger breaks the contact with the surface by detecting when the finger changes from the second state of being in contact with the surface back to the first state of not being in contact with the surface; and
identifying a touch interaction based on the detected events, wherein identifying the touch interaction comprises identifying a type of the touch interaction and is based on analyzing the plurality of channels of the deformation data, wherein the characteristics are used to identify the type of the touch interaction, identify the duration of the touch interaction, and identify an amount of movement associated with the touch interaction.
2 . The method of claim 1 , wherein detecting the events is based on an estimation of displacement of the finger.
3 . The method of claim 1 , wherein detecting the events is based on at least four measures of displacements of the finger based on at least four deformation sensors.
4 . The method of claim 1 , wherein detecting the events based on the estimation of displacement of the finger comprises accounting for a baseline displacement of the finger.
5 . The method of claim 1 , wherein detecting the events is based on an estimation of velocity of the displacement.
6 . The method of claim 1 further comprising determining a measurement of a physical interaction of the finger based on the deformation data, wherein the touch interaction is identified based on the measurement.
7 . The method of claim 6 , wherein the measurement comprises a force, direction, or shear.
8 . The method of claim 1 , wherein identifying the touch interaction comprises identifying a touch stage state.
9 . The method of claim 1 , wherein identifying the touch interaction comprises identifying that a type of the touch interaction is:
a pinch of the finger against another finger;
a tap of the finger on a surface; or
a swipe of the finger on the surface.
10 . The method of claim 9 , wherein identifying the type of the touch interaction comprises identifying whether the pinch is a long pinch or quick pinch based on a time threshold.
11 . The method of claim 9 , wherein identifying the type of the touch interaction comprises identifying whether the touch interaction is a single tap or a double tap based on a time threshold.
12 . A system comprising:
a non-transitory computer-readable storage medium; and
one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:
obtaining a plurality of channels of deformation data from a plurality of sensors on a device, wherein the sensors are positioned adjacent to a plurality of portions of a finger to detect an amount of change in position of portions of skin corresponding to deformations of the plurality of portions of the finger, wherein the deformation data comprises time-based data corresponding to detecting the deformations of the plurality of portions of the finger over time;
detecting events based on the deformation data, wherein the events are detected by analyzing time segments of data from the plurality of channels and identifying characteristics of the time segments to distinguish different types of touch interactions, wherein detecting the events comprising:
detecting a first event during which the finger makes contact with a surface by detecting when the finger changes from a first state of not being in contact with the surface to a second state of being in contact with the surface; and
detecting a second event, following the first event, during which the finger breaks the contact with the surface by detecting when the finger changes from the second state of being in contact with the surface back to the first state of not being in contact with the surface; and
identifying a touch interaction based on the detected events, wherein identifying the touch interaction comprises identifying a type of the touch interaction and is based on analyzing the plurality of channels of the deformation data, wherein the characteristics are used to identify the type of the touch interaction, identify the duration of the touch interaction, and identify an amount of movement associated with the touch interaction.
13 . The system of claim 12 , wherein detecting the events is based on an estimation of displacement of the finger and a rate of displacement.
14 . The system of claim 12 , wherein detecting the events is based on at least four measures of displacements of the finger based on at least four deformation sensors.
15 . The system of claim 13 , wherein the operations further comprise determining a measurement of a physical interaction of the finger based on the deformation data, wherein the touch interaction is identified based on the measurement.
16 . A non-transitory computer-readable storage medium, storing program instructions executable by one or more processors to perform operations comprising:
obtaining a plurality of channels of deformation data from a plurality of sensors on a device, wherein the sensors are positioned adjacent to a plurality of portions of a finger to detect an amount of change in position of portions of skin corresponding to deformations of the plurality of portions of the finger, wherein the deformation data comprises time-based data corresponding to detecting the deformations of the plurality of portions of the finger over time;
detecting events based on the deformation data, wherein the events are detected by analyzing time segments of data from the plurality of channels and identifying characteristics of the time segments to distinguish different types of touch interactions, wherein detecting the events comprising:
detecting a first event during which the finger makes contact with a surface by detecting when the finger changes from a first state of not being in contact with the surface to a second state of being in contact with the surface; and
detecting a second event, following the first event, during which the finger breaks the contact with the surface by detecting when the finger changes from the second state of being in contact with the surface back to the first state of not being in contact with the surface; and
identifying a touch interaction based on the detected events, wherein identifying the touch interaction comprises identifying a type of the touch interaction and is based on analyzing the plurality of channels of the deformation data, wherein the characteristics are used to identify the type of the touch interaction, identify the duration of the touch interaction, and identify an amount of movement associated with the touch interaction.
17 . The non-transitory computer-readable storage medium of claim 16 , wherein detecting the events is based on an estimation of displacement of the finger and a rate of displacement.
18 . The non-transitory computer-readable storage medium of claim 16 , wherein the operations further comprise determining a measurement of a physical interaction of the finger based on the deformation data, wherein the touch interaction is identified based on the measurement.