IMU for touch detection
Touch detection may include determining, based on data from an IMU on a first device that monitors movement of a touching object, a touch event, wherein the touch event indicates contact between the touching object and a surface, obtaining a depth image captured by a second device, comprising a touch image, determining a touch point of the object based on the touch image, and providing a response based on the touch point of the object and the touched surface.
1. A non-transitory computer readable medium comprising computer readable code executable by one or more processors to:
detect, at a first device, an indication that a touch event has occurred based on data received from an inertial measurement unit (IMU) on a second device that monitors movement of a touching object, wherein the touch event indicates contact between the touching object and a surface;
in response to a touch event:
obtain, by the first device, a depth image comprising a touch image from the second device;
determine a touch point on the surface based on the touch image and the touching object and surface in the image; and
provide a response based on the touch point.
2. The non-transitory computer readable medium of claim 1 , wherein the touching object comprises a fingertip and wherein the touch event occurs between the fingertip and the surface.
3. The non-transitory computer readable medium of claim 1 , wherein the computer readable code to detect the indication that the touch event has occurred further comprise computer readable code to:
determine that the touch event is a true touch event based on depth information from the touch image for the object and the surface.
4. The non-transitory computer readable medium of claim 1 , further comprising computer readable code to:
detect, by the IMU, a change in direction of the object; and
capture a non-touch image in response to the detected change in direction of the object.
5. The non-transitory computer readable medium of claim 1 , further comprising computer readable code to:
obtain a non-touch image; and
cause a model of the touching object to be generated based on the touch image, the non-touch image, and the touch point.
6. The non-transitory computer readable medium of claim 5 , further comprising computer readable code to:
provide the model of the touching object to the second device for visual tracking of the touching object.
7. The non-transitory computer readable medium of claim 1 , wherein the second device is a wearable device.
8. A system for providing touch detection, comprising:
one or more processors; and
a memory coupled to the one or more processors and comprising computer readable code executable by the one or more processors to:
detect, at a first device, an indication that a touch event has occurred based on data received from an inertial measurement unit (IMU) on a second device that monitors movement of a touching object, wherein the touch event indicates contact between the touching object and a surface;
in response to a touch event:
obtain, by the first device, a depth image comprising a touch image from the second device;
determine a touch point on the surface based on the touch image and the touching object and surface in the image; and
provide a response based on the touch point.
9. The system of claim 8 , wherein the touching object comprises a fingertip and wherein the touch event occurs between the fingertip and the surface.
10. The system of claim 8 , wherein the computer readable code to detect the indication that the touch event has occurred further comprise computer readable code to:
determine that the touch event is a true touch event based on depth information from the touch image for the object and the surface.
11. The system of claim 8 , further comprising computer readable code to:
detect, by the IMU, a change in direction of the object; and
capture a non-touch image in response to the detected change in direction of the object.
12. The system of claim 8 , further comprising computer readable code to:
obtain a non-touch image; and
cause a model of the touching object to be generated based on the touch image, the non-touch image, and the touch point.
13. The system of claim 12 , further comprising computer readable code to:
provide the model of the touching object to the second device for visual tracking of the touching object.
14. The system of claim 8 , wherein the second device is a wearable device.
15. A method for touch detection, comprising:
detecting, at a first device, an indication that a touch event has occurred based on data received from an inertial measurement unit (IMU) on a second device that monitors movement of a touching object, wherein the touch event indicates contact between the touching object and a surface;
in response to a touch event:
obtaining, by the first device, a depth image comprising a touch image from the second device;
determining a touch point on the surface based on the touch image and the touching object and surface in the image; and
providing a response based on the touch point.
16. The method of claim 15 , wherein the touching object comprises a fingertip and wherein the touch event occurs between the fingertip and the surface.
17. The method of claim 15 , wherein detecting the indication that the touch event has occurred comprises:
determining that the touch event is a true touch event based on depth information from the touch image for the object and the surface.
18. The method of claim 15 , further comprising:
detecting, by the IMU, a change in direction of the object; and
capturing a non-touch image in response to the detected change in direction of the object.
19. The method of any of claim 15 , further comprising:
obtaining a non-touch image; and
causing a model of the touching object to be generated based on the touch image, the non-touch image, and the touch point.
20. The method of claim 19 , further comprising:
providing the model of the touching object to the second device for visual tracking of the touching object.